BayesianOptimizer#

class obsidian.optimizer.bayesian.BayesianOptimizer(X_space: ParamSpace, surrogate: str | dict | list[str] | list[dict] = 'GP', task: TaskType | str = TaskType.OPTIMIZATION, seed: int | None = None, rng: RNGManager | None = None, fix_random_state: bool = True, verbose: int = 1)[source]#

Bases: Optimizer

BayesianOptimizer is a class that implements a Bayesian optimization algorithm.

This class is used to optimize a given function by iteratively selecting the next set of input parameters based on the results of previous evaluations. It uses a surrogate model to approximate the underlying function and an acquisition function to determine the next set of parameters to evaluate.

Parameters:

X_space (ParamSpace) – The parameter space defining the search space for the optimization.
surrogate (str | dict | list[str] | list[dict], optional) –
The surrogate model(s) to use. It can be a string representing a single model type, a dictionary specifying multiple model types with their hyperparameters, or a list of strings or dictionaries.

Defaults to 'GP'. Options are as follows:
- 'GP': Gaussian Process with default settings (Matern Kernel, Gamma covariance priors)
- 'MixedGP': GP with mixed parameter types (continuous, categorical). Will be re-selected by default if ‘GP’ is selected and input space is mixed.
- 'DKL': GP with a NN feature-extractor (deep kernel learning)
- 'GPflat': GP without priors. May result in optimization instability, but removes bias for special situations.
- 'GPprior': GP with custom priors on the mean, likelihood, and covariance
- 'MTGP': Multi-task GP for multi-output optimization. Will be re-selected by default if ‘GP’ is selected and the input space contains Task parameters.
- 'DNN': Dropout neural network. Uses MC sampling to mask neurons during training and to estimate uncertainty.
seed (int | None, optional) – The random seed to use. Defaults to None.
verbose (int, optional) – The verbosity level. Defaults to 1.

surrogate_type#

The shorthand name of each surrogate model.

Type:: list[str]

surrogate_hps#

The hyperparameters for each surrogate model.

Type:: list[dict]

is_fit#

Indicates whether the surrogate model has been fit to data.

Type:: bool

Raises:

TypeError – If the surrogate argument is not a string, dict, or list of str/dict.
ValueError – If the surrogate dictionary contains more than one surrogate model type.
KeyError – If the surrogate model is not selected from the available models.
ValueError – If the number of responses does not match the number of specified surrogate

__init__(X_space: ParamSpace, surrogate: str | dict | list[str] | list[dict] = 'GP', task: TaskType | str = TaskType.OPTIMIZATION, seed: int | None = None, rng: RNGManager | None = None, fix_random_state: bool = True, verbose: int = 1)[source]#

Methods

`__init__`(X_space[, surrogate, task, seed, ...])
`evaluate`(X_suggest[, X_t_pending, target, ...])
`fit`(Z, target[, manual_seed, fit_options])	Fit surrogate model(s) using observed data.
`hypervolume`(f[, ref_point, weights])	Calculates the hypervolume of the given data points.
`load_state`(config_save)	Loads the parameters of the Bayesian Optimizer from a saved state.
`maximize`([acquisition, optim_samples, ...])	Predicts the conditions which return the maximum response value within the parameter space.
`pareto`(f)	Determines the Pareto dominance of a given set of solutions.
`pf_distance`(y)	Calculates the pairwise distance between the given input y and the Pareto front.
`predict`(X[, return_f_inv, PI_range])	Predicts a response over a range of experiments using the surrogate function.
`save_state`()	Saves the parameters of the Bayesian Optimizer so that they can be reloaded.
`set_X_space`(X_space)
`suggest`([m_batch, target, acquisition, ...])	Suggest future experiments based on a maximization of some acquisition function calculated from the expectation of a surrogate model.

Attributes

`X_space`	The parameter space defining the search space for the optimization.
`is_fit`	Check if all surrogate mdoels in optimizer are fit
`non_tracking_targets`	Targets eligible for suggestion (all non-tracking targets).

Parameters:

X_suggest (pd.DataFrame) – Experiment matrix of real input variables, selected by optimizer.
X_t_pending (Tensor) – Suggested experiments yet to be run
target (Target or list of Target, optional) – The response(s) to be used for optimization,
acquisition (str | dict, optional) – Acquisition function name (str) or dictionary containing the acquisition function name and its hyperparameters.
objective (MCAcquisitionObjective, optional) – The objective function to be used for optimization. The default is None.
eval_aq (bool, optional) – Whether or not to also evaluate the aq function. The default is False.
manual_seed (int | None, optional) – Seed applied to the acquisition evaluation (when eval_aq is True) so the reported aq value is reproducible and consistent with the optimized candidate. If None, a seed is resolved from the optimizer’s RNG state. Defaults to None.

Returns:

Response prediction, pred interval, transformed mean, aq value,: and objective function evaluation(s)

Return type:

pd.DataFrame

property is_fit#

Check if all surrogate mdoels in optimizer are fit

Returns:: True if the optimizer is fit, False otherwise.
Return type:: bool

classmethod load_state(config_save: dict)[source]#

Loads the parameters of the Bayesian Optimizer from a saved state.

Can load both fitted and unfitted optimizers.

Parameters:: config_save (dict) – A dictionary containing the saved state.
Returns:: Loaded optimizer instance
Return type:: BayesianOptimizer
Raises:: ValueError – If the number of saved models does not match the number of named models.

maximize(acquisition=['Mean'], optim_samples=1026, optim_restarts=50, fixed_var: dict[str, float | str] | None = None) → tuple[DataFrame, DataFrame][source]#

Predicts the conditions which return the maximum response value within the parameter space.

Parameters:

optim_samples (int) – The number of samples to be used for optimization. Default is 1026.
optim_restarts (int) – The number of restarts for the optimization process. Default is 50.
(dict(str (fixed_var) – float), optional): Name of a variable and setting, over which the suggestion should be fixed. Default values is None

Returns:

tuple[pd.DataFrame, pd.DataFrame] = (X_suggest, eval_suggest)

X_suggest (pd.DataFrame): Experiment matrix of real input variables,: selected by optimizer.
y_suggest (pd.DataFrame): Mean results and prediction interval for: each suggested experiment.

property non_tracking_targets: list[Target]#: Targets eligible for suggestion (all non-tracking targets).

predict(X: DataFrame, return_f_inv: bool = True, PI_range: float = 0.7) → DataFrame[source]#

Predicts a response over a range of experiments using the surrogate function.

Parameters:

X (pd.DataFrame) – Experiments to predict over.
return_f_inv (bool, optional) – Whether or not to return the inverse-transformed objective function, which is the raw response (unscored). The default is True. Most internal calls set to False to handle the transformed objective function.
PI_range (float, optional) – The nominal coverage range for the returned prediction interval

Returns:

Mean prediction and prediction interval for each response

Return type:

pd.DataFrame

Raises:

TypeError – If the input is not a DataFrame.
UnfitError – If the surrogate model has not been fit before predicting.
ValueError – If the prediction interval range is greater than 1.
NameError – If the input does not contain all of the required predictors from the training set.

save_state() → dict[source]#

Saves the parameters of the Bayesian Optimizer so that they can be reloaded.

This method can save both fitted and unfitted optimizers:

Unfitted: Saves configuration only (X_space, surrogate spec, RNG state)
Fitted: Saves configuration + training data + fitted models

Returns:: A dictionary containing the optimizer state for later loading.
Return type:: dict

Examples

>>> # Save unfitted optimizer
>>> opt = BayesianOptimizer(X_space, seed=42)
>>> state = opt.save_state()  # Works without fitting

>>> # Load and then fit
>>> opt2 = BayesianOptimizer.load_state(state)
>>> opt2.fit(data, target=target)

Suggest future experiments based on a maximization of some acquisition function calculated from the expectation of a surrogate model.

Parameters:

m_batch (int, optional) – The number of experiments to suggest at once. The default is 1.
target (Target or list of Target, optional) – The response(s) to be used for optimization.
acquisition (list of str or list of dict, optional) –
Indicator for the desired acquisition function(s). A list will propose experiments for each acquisition function based on optim_sequential.

The default is ['NEI'] for single-output and ['NEHVI'] for multi-output. Options are as follows:
- 'EI': Expected Improvement (relative to best of y_train). Accepts hyperparameter 'inflate', a positive or negative float to inflate/deflate the best point for explore/exploit.
- 'NEI': Noisy Expected Improvement. More robust than EI and uses all of y_train, but accepts no hyperparameters.
- 'PI': Probability of Improvement (relative to best of y_train). Accepts hyperparameter 'inflate', a positive or negative float to inflate/deflate the best point for explore/exploit.
- 'UCB': Upper Confidence Bound. Accepts hyperparameter 'beta', a positive float which sets the number of standard deviations above the mean.
- 'SR': Simple Regret
- 'RS': Random Sampling
- 'Mean': Mean of the posterior distribution (pure exploitation/maximization of objective)
- 'SF': Space Filling. Requests points that maximize the minimumd distance to X_train based on Euclidean distance.
- 'NIPV': Negative Integrated Posterior Variance. Requests the point which most improves the prediction interval for a random selection of points in the design space. Used for active learning.
- 'EHVI': Expected Hypervolume Improvement. Can accept a ref_point, otherwise a point just below the minimum of y_train.
- 'NEHVI': Noisy Expected Hypervolume Improvement. Can accept a ref_point, otherwise a point just below the minimum of y_train.
- 'NParEGO': Noisy Pareto Efficient Global Optimization. Can accept scalarization_weights, a list of weights for each objective.
optim_sequential (bool, optional) – Whether or not to optimize batch designs sequentially (by fantasy) or simultaneously. Default is True.
optim_samples (int, optional) – The number of samples to use for quasi Monte Carlo sampling of the acquisition function. Also used for initializing the acquisition optimizer. The default value is 512.
optim_restarts (int, optional) – The number of restarts to use in the global optimization of the acquisition function. The default value is 10.
optim_options (dict, optional) – Options to pass to the optimization routine directly. Refer to BoTorch’s optimize_acqf function family, gen_candidates_scipy, gen_candidates_torch, and scipy.optimize.minimize for possible options.
objective (MCAcquisitionObjective, optional) – The objective function to be used for optimization. The default is None.
out_constraints (Output_Constraint | list[Output_Constraint], optional) – An output constraint, or a list thereof, restricting the search space by outcomes. The default is None.
eq_constraints (Linear_Constraint | list[Linear_Constraint], optional) – A linear constraint, or a list thereof, restricting the search space by equality (=). The default is None.
ineq_constraints (Linear_Constraint | list[Linear_Constraint], optional) – A linear constraint, or a list thereof, restricting the search space by inequality (>=). The default is None.
nleq_constraints (Nonlinear_Constraint | list[Nonlinear_Constraint], optional) – A nonlinear constraint, or a list thereof, restricting the search space by nonlinear feasibility. The default is None.
task_index (int, optional) – The index of the task to optimize for multi-task models. The default is 0.
(dict(str (fixed_var) – float), optional): Name of a variable and setting, over which the suggestion should be fixed. Default values is None
X_pending (pd.DataFrame, optional) – Experiments that are expected to be run before the next optimal set
eval_pending (pd.DataFrame, optional) – Acquisition values associated with X_pending

Returns:

tuple[pd.DataFrame, pd.DataFrame] = (X_suggest, eval_suggest)

X_suggest (pd.DataFrame): Experiment matrix of real input variables,: selected by optimizer.
eval_suggest (pd.DataFrame): Mean results (response, prediction interval, f(response), obj: function for each suggested experiment.

Raises:

UnfitError – If the surrogate model has not been fit before suggesting new experiments.
TypeError – If the target is not a Target object or a list of Target objects.
IncorrectObjectiveError – If the objective does not successfully execute on a sample.
TypeError – If the acquisition is not a list of strings or dictionaries.
UnsupportedError – If the provided acquisition function does not support output constraints.