Skip to content
Snippets Groups Projects
Commit e0854566 authored by Yogeswaran Umasankar's avatar Yogeswaran Umasankar
Browse files

New upstream version 1.5.1

parent 1599df99
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 102 additions and 40 deletions
.github/workflows/build_docs.yml
+ 5
− 11
View file @ e0854566
......@@ -22,21 +22,15 @@ jobs:
python-version: '3.10'
- name: Install pandoc
run: sudo apt-get install -y pandoc
- name: Install dependencies
- name: Install Poetry
uses: snok/install-poetry@v1
- name: Install package and test dependencies
run: |
python -m pip install --upgrade pip
pip install sphinx
pip install nbsphinx
pip install sphinx_rtd_theme
pip install jupyter
pip install myst-parser
- name: Install package
run: |
pip install -e .
poetry install --with dev,nbtools
- name: build sphinx docs
run: |
cd docsrc
make github
poetry run make github
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
if: ${{ github.ref == 'refs/heads/master' }}
......
.github/workflows/run_tests.yml
+ 3
− 1
View file @ e0854566
......@@ -13,11 +13,12 @@ permissions:
jobs:
build:
name: Python ${{ matrix.python-version }} - numpy ${{ matrix.numpy-version }}
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
numpy-version: [">=1.25,<2", ">=2"]
steps:
- uses: actions/checkout@v3
......@@ -29,6 +30,7 @@ jobs:
uses: snok/install-poetry@v1
- name: Install test dependencies
run: |
poetry add "numpy${{ matrix.numpy-version }}"
poetry install --with dev,nbtools
- name: Test with pytest
run: |
......
README.md
+ 5
− 5
View file @ e0854566
<div align="center">
<img src="https://raw.githubusercontent.com/bayesian-optimization/BayesianOptimization/master/static/func.png"><br><br>
<img src="https://raw.githubusercontent.com/bayesian-optimization/BayesianOptimization/master/docsrc/static/func.png"><br><br>
</div>
# Bayesian Optimization
![tests](https://github.com/bayesian-optimization/BayesianOptimization/actions/workflows/run_tests.yml/badge.svg)
[![Codecov](https://codecov.io/github/bayesian-optimization/BayesianOptimization/badge.svg?branch=master&service=github)](https://codecov.io/github/bayesian-optimization/BayesianOptimization?branch=master)
[![Pypi](https://img.shields.io/pypi/v/bayesian-optimization.svg)](https://pypi.python.org/pypi/bayesian-optimization)
[![Pypi](https://img.shields.io/pypi/v/bayesian-optimization.svg)](https://pypi.python.org/pypi/bayesian-optimization)![PyPI - Python Version](https://img.shields.io/pypi/pyversions/bayesian-optimization)
Pure Python implementation of bayesian global optimization with gaussian
processes.
......@@ -51,11 +51,11 @@ for ideas on how to implement bayesian optimization in a distributed fashion usi
Bayesian optimization works by constructing a posterior distribution of functions (gaussian process) that best describes the function you want to optimize. As the number of observations grows, the posterior distribution improves, and the algorithm becomes more certain of which regions in parameter space are worth exploring and which are not, as seen in the picture below.
![BayesianOptimization in action](./static/bo_example.png)
![BayesianOptimization in action](docsrc/static/bo_example.png)
As you iterate over and over, the algorithm balances its needs of exploration and exploitation taking into account what it knows about the target function. At each step a Gaussian Process is fitted to the known samples (points previously explored), and the posterior distribution, combined with a exploration strategy (such as UCB (Upper Confidence Bound), or EI (Expected Improvement)), are used to determine the next point that should be explored (see the gif below).
![BayesianOptimization in action](./static/bayesian_optimization.gif)
![BayesianOptimization in action](docsrc/static/bayesian_optimization.gif)
This process is designed to minimize the number of steps required to find a combination of parameters that are close to the optimal combination. To do so, this method uses a proxy optimization problem (finding the maximum of the acquisition function) that, albeit still a hard problem, is cheaper (in the computational sense) and common tools can be employed. Therefore Bayesian Optimization is most adequate for situations where sampling the function to be optimized is a very expensive endeavor. See the references for a proper discussion of this method.
......@@ -183,7 +183,7 @@ Sometimes the initial boundaries specified for a problem are too wide, and addin
When it's worthwhile to converge on an optimal point quickly rather than try to find the optimal point, contracting the domain around the current optimal value as the search progresses can speed up the search progress considerably. Using the `SequentialDomainReductionTransformer` the bounds of the problem can be panned and zoomed dynamically in an attempt to improve convergence.
![sequential domain reduction](./static/sdr.png)
![sequential domain reduction](docsrc/static/sdr.png)
An example of using the `SequentialDomainReductionTransformer` is shown in the [domain reduction notebook](http://bayesian-optimization.github.io/BayesianOptimization/domain_reduction.html). More information about this method can be found in the paper ["On the robustness of a simple domain reduction scheme for simulation‐based optimization"](http://www.truegrid.com/srsm_revised.pdf).
......
bayes_opt/logger.py
+ 12
− 5
View file @ e0854566
......@@ -2,6 +2,7 @@
from __future__ import print_function
import os
import json
import numpy as np
from .observer import _Tracker
from .event import Events
from colorama import Fore, just_fix_windows_console
......@@ -45,7 +46,10 @@ class ScreenLogger(_Tracker):
_default_cell_size = 9
_default_precision = 4
_colour_new_max = Fore.MAGENTA
_colour_regular_message = Fore.RESET
_colour_reset = Fore.RESET
def __init__(self, verbose=2, is_constrained=False):
self._verbose = verbose
self._is_constrained = is_constrained
......@@ -135,7 +139,7 @@ class ScreenLogger(_Tracker):
return s[:self._default_cell_size - 3] + "..."
return s
def _step(self, instance, colour=Fore.BLACK):
def _step(self, instance, colour=_colour_regular_message):
"""Log a step.
Parameters
......@@ -144,7 +148,7 @@ class ScreenLogger(_Tracker):
The instance associated with the event.
colour :
(Default value = Fore.BLACK)
(Default value = _colour_regular_message, equivalent to Fore.RESET)
Returns
-------
......@@ -162,7 +166,7 @@ class ScreenLogger(_Tracker):
for key in instance.space.keys:
cells.append(self._format_number(res["params"][key]))
return "| " + " | ".join([colour + cells[i] for i in range(len(cells))]) + " |"
return "| " + " | ".join([colour + cells[i] + self._colour_reset for i in range(len(cells))]) + " |"
def _header(self, instance):
"""Print the header of the log.
......@@ -230,7 +234,7 @@ class ScreenLogger(_Tracker):
if self._verbose == 1 and not is_new_max:
line = ""
else:
colour = Fore.MAGENTA if is_new_max else Fore.BLACK
colour = self._colour_new_max if is_new_max else self._colour_regular_message
line = self._step(instance, colour=colour) + "\n"
elif event == Events.OPTIMIZATION_END:
line = "=" * self._header_length + "\n"
......@@ -291,6 +295,9 @@ class JSONLogger(_Tracker):
if "allowed" in data: # fix: github.com/fmfn/BayesianOptimization/issues/361
data["allowed"] = bool(data["allowed"])
if "constraint" in data and isinstance(data["constraint"], np.ndarray):
data["constraint"] = data["constraint"].tolist()
with open(self._path, "a") as f:
f.write(json.dumps(data) + "\n")
......
bayes_opt/target_space.py
+ 16
− 7
View file @ e0854566
......@@ -302,7 +302,7 @@ class TargetSpace():
self.n_duplicate_points = self.n_duplicate_points + 1
print(Fore.RED + f'Data point {x} is not unique. {self.n_duplicate_points}'
' duplicates registered. Continuing ...')
' duplicates registered. Continuing ...' + Fore.RESET)
else:
raise NotUniqueError(f'Data point {x} is not unique. You can set'
' "allow_duplicate_points=True" to avoid this error')
......@@ -312,21 +312,30 @@ class TargetSpace():
if not np.all((self._bounds[:, 0] <= x) & (x <= self._bounds[:, 1])):
warn(f'\nData point {x} is outside the bounds of the parameter space. ', stacklevel=2)
self._params = np.concatenate([self._params, x.reshape(1, -1)])
self._target = np.concatenate([self._target, [target]])
# Make copies of the data, so as not to modify the originals incase something fails
# during the registration process. This prevents out-of-sync data.
params_copy = np.concatenate([self._params, x.reshape(1, -1)])
target_copy = np.concatenate([self._target, [target]])
cache_copy = self._cache.copy() # shallow copy suffices
if self._constraint is None:
# Insert data into unique dictionary
self._cache[_hashable(x.ravel())] = target
cache_copy[_hashable(x.ravel())] = target
else:
if constraint_value is None:
msg = ("When registering a point to a constrained TargetSpace" +
" a constraint value needs to be present.")
raise ValueError(msg)
# Insert data into unique dictionary
self._cache[_hashable(x.ravel())] = (target, constraint_value)
self._constraint_values = np.concatenate([self._constraint_values,
[constraint_value]])
cache_copy[_hashable(x.ravel())] = (target, constraint_value)
constraint_values_copy = np.concatenate([self._constraint_values,
[constraint_value]])
self._constraint_values = constraint_values_copy
# Operations passed, update the variables
self._params = params_copy
self._target = target_copy
self._cache = cache_copy
def probe(self, params):
"""Evaluate the target function on a point and register the result.
......
docsrc/Makefile
+ 2
− 2
View file @ e0854566
......@@ -15,10 +15,10 @@ BUILDDIR = ../docs
.PHONY: help Makefile
github:
@cp ../README.md .
@cp -r ../static .
# @cp ../README.md .
@make html
@cp -a ../docs/html/. ../docs
@cp -r ../docsrc/ ../docs
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
......
docsrc/quickstart.rst
+ 1
− 1
View file @ e0854566
.. include:: README.md
.. include:: ../README.md
:parser: myst_parser.sphinx_
\ No newline at end of file
static/bo_example.png → docsrc/static/bo_example.png
+ 0
− 0
View file @ e0854566
File moved
static/func.png → docsrc/static/func.png
+ 0
− 0
View file @ e0854566
File moved
static/sdr.png → docsrc/static/sdr.png
+ 0
− 0
View file @ e0854566
File moved
examples/advanced-tour.ipynb
+ 3
− 3
View file @ e0854566
%% Cell type:markdown id: tags:
# Advanced tour of the Bayesian Optimization package
%% Cell type:code id: tags:
``` python
from bayes_opt import BayesianOptimization
```
%% Cell type:markdown id: tags:
## 1. Suggest-Evaluate-Register Paradigm
Internally the `maximize` method is simply a wrapper around the methods `suggest`, `probe`, and `register`. If you need more control over your optimization loops the Suggest-Evaluate-Register paradigm should give you that extra flexibility.
For an example of running the `BayesianOptimization` in a distributed fashion (where the function being optimized is evaluated concurrently in different cores/machines/servers), checkout the `async_optimization.py` script in the examples folder.
%% Cell type:code id: tags:
``` python
# Let's start by defining our function, bounds, and instantiating an optimization object.
def black_box_function(x, y):
return -x ** 2 - (y - 1) ** 2 + 1
```
%% Cell type:markdown id: tags:
Notice that the evaluation of the blackbox function will NOT be carried out by the optimizer object. We are simulating a situation where this function could be being executed in a different machine, maybe it is written in another language, or it could even be the result of a chemistry experiment. Whatever the case may be, you can take charge of it and as long as you don't invoke the `probe` or `maximize` methods directly, the optimizer object will ignore the blackbox function.
%% Cell type:code id: tags:
``` python
optimizer = BayesianOptimization(
f=None,
pbounds={'x': (-2, 2), 'y': (-3, 3)},
verbose=2,
random_state=1,
)
```
%% Cell type:markdown id: tags:
One extra ingredient we will need is an `UtilityFunction` instance. In case it is not clear why, take a look at the literature to understand better how this method works.
%% Cell type:code id: tags:
``` python
from bayes_opt import UtilityFunction
utility = UtilityFunction(kind="ucb", kappa=2.5, xi=0.0)
```
%% Cell type:markdown id: tags:
The `suggest` method of our optimizer can be called at any time. What you get back is a suggestion for the next parameter combination the optimizer wants to probe.
Notice that while the optimizer hasn't observed any points, the suggestions will be random. However, they will stop being random and improve in quality the more points are observed.
%% Cell type:code id: tags:
``` python
next_point_to_probe = optimizer.suggest(utility)
print("Next point to probe is:", next_point_to_probe)
```
%% Output
Next point to probe is: {'x': -0.331911981189704, 'y': 1.3219469606529488}
%% Cell type:markdown id: tags:
You are now free to evaluate your function at the suggested point however/whenever you like.
%% Cell type:code id: tags:
``` python
target = black_box_function(**next_point_to_probe)
print("Found the target value to be:", target)
```
%% Output
Found the target value to be: 0.7861845912690542
%% Cell type:markdown id: tags:
Last thing left to do is to tell the optimizer what target value was observed.
%% Cell type:code id: tags:
``` python
optimizer.register(
params=next_point_to_probe,
target=target,
)
```
%% Cell type:markdown id: tags:
### 1.1 The maximize loop
And that's it. By repeating the steps above you recreate the internals of the `maximize` method. This should give you all the flexibility you need to log progress, hault execution, perform concurrent evaluations, etc.
%% Cell type:code id: tags:
``` python
for _ in range(5):
next_point = optimizer.suggest(utility)
target = black_box_function(**next_point)
optimizer.register(params=next_point, target=target)
print(target, next_point)
print(optimizer.max)
```
%% Output
-18.49187152919165 {'x': 1.8861546000771092, 'y': -2.9917780942581977}
0.7911494590443674 {'x': -0.31764604716962586, 'y': 1.3285597809731806}
-6.999999999999999 {'x': -1.9999999999999998, 'y': 3.0}
-7.0 {'x': 2.0, 'y': 3.0}
-7.503866814436659 {'x': -2.0, 'y': -1.1222315647536345}
{'target': 0.7911494590443674, 'params': {'x': -0.31764604716962586, 'y': 1.3285597809731806}}
%% Cell type:markdown id: tags:
## 2. Dealing with discrete parameters
**There is no principled way of dealing with discrete parameters using this package.**
Ok, now that we got that out of the way, how do you do it? You're bound to be in a situation where some of your function's parameters may only take on discrete values. Unfortunately, the nature of bayesian optimization with gaussian processes doesn't allow for an easy/intuitive way of dealing with discrete parameters - but that doesn't mean it is impossible. The example below showcases a simple, yet reasonably adequate, way to dealing with discrete parameters.
%% Cell type:code id: tags:
``` python
def func_with_discrete_params(x, y, d):
# Simulate necessity of having d being discrete.
assert type(d) == int
return ((x + y + d) // (1 + d)) / (1 + (x + y) ** 2)
```
%% Cell type:code id: tags:
``` python
def function_to_be_optimized(x, y, w):
d = int(w)
return func_with_discrete_params(x, y, d)
```
%% Cell type:code id: tags:
``` python
optimizer = BayesianOptimization(
f=function_to_be_optimized,
pbounds={'x': (-10, 10), 'y': (-10, 10), 'w': (0, 5)},
verbose=2,
random_state=1,
)
```
%% Cell type:code id: tags:
``` python
optimizer.set_gp_params(alpha=1e-3)
optimizer.maximize()
```
%% Output
| iter | target | w | x | y |
-------------------------------------------------------------
| 1  | -0.06199  | 2.085  | 4.406  | -9.998  |
| 2  | -0.0344  | 1.512  | -7.065  | -8.153  |
| 3  | -0.2177  | 0.9313  | -3.089  | -2.065  |
| 4  | 0.1865  | 2.694  | -1.616  | 3.704  |
| 5  | -0.2187  | 1.022  | 7.562  | -9.452  |
| 6  | 0.1868  | 2.533  | -1.728  | 3.815  |
| 7  | 0.05119  | 3.957  | -0.6151  | 6.785  |
| 8  | 0.1761  | 0.5799  | 1.181  | 4.054  |
| 9  | 0.04045  | 4.004  | 4.304  | 2.656  |
| 10  | 0.07509  | 0.0  | 4.843  | 7.759  |
| 11  | 0.3512  | 0.0  | -5.713  | 7.072  |
| 12  | -0.8068  | 0.0  | -9.09  | 8.6  |
| 13  | 0.3774  | 0.3974  | -4.19  | 6.264  |
| 14  | 0.157  | 0.0  | -3.587  | 8.534  |
| 15  | -0.7891  | 0.4794  | -5.536  | 4.298  |
| 16  | 0.1176  | 1.038  | -4.671  | 7.41  |
| 17  | 0.1815  | 0.4815  | -2.66  | 6.6  |
| 18  | 0.08677  | 1.933  | -0.1438  | 4.839  |
| 19  | 0.1687  | 1.139  | -0.4707  | 2.69  |
| 20  | 0.1133  | 2.363  | 1.344  | 2.736  |
| 21  | 0.2401  | 0.0  | 1.441  | 1.949  |
| 22  | 0.1568  | 0.1832  | 3.2  | 2.904  |
| 23  | 0.2722  | 0.9731  | 2.625  | 0.5406  |
| 24  | 0.0  | 1.149  | 0.7191  | 0.2267  |
| 25  | 0.1686  | 0.0  | 4.181  | 0.5867  |
| 26  | 0.0644  | 2.276  | 3.975  | -0.1631  |
| 27  | 0.4397  | 0.08737  | 2.66  | -1.531  |
| 28  | 0.2904  | 0.0  | 3.913  | -2.35  |
| 29  | -0.9874  | 0.0  | 1.992  | -3.005  |
| 30  | 0.3001  | 0.2116  | 3.375  | -0.9955  |
=============================================================
%% Cell type:markdown id: tags:
## 3. Tuning the underlying Gaussian Process
The bayesian optimization algorithm works by performing a gaussian process regression of the observed combination of parameters and their associated target values. The predicted parameter$\rightarrow$target hyper-surface (and its uncertainty) is then used to guide the next best point to probe.
%% Cell type:markdown id: tags:
### 3.1 Passing parameter to the GP
Depending on the problem it could be beneficial to change the default parameters of the underlying GP. You can use the `optimizer.set_gp_params` method to do this:
%% Cell type:code id: tags:
``` python
optimizer = BayesianOptimization(
f=black_box_function,
pbounds={'x': (-2, 2), 'y': (-3, 3)},
verbose=2,
random_state=1,
)
optimizer.set_gp_params(alpha=1e-3, n_restarts_optimizer=5)
optimizer.maximize(
init_points=1,
n_iter=5
)
```
%% Output
| iter | target | x | y |
-------------------------------------------------
| 1  | 0.7862  | -0.3319  | 1.322  |
| 2  | -18.49  | 1.886  | -2.992  |
| 3  | 0.7911  | -0.3176  | 1.329  |
| 4  | -6.11  | -1.763  | 3.0  |
| 5  | -2.895  | 1.533  | 2.243  |
| 6  | -4.806  | -2.0  | -0.3439  |
=================================================
%% Cell type:markdown id: tags:
### 3.2 Tuning the `alpha` parameter
When dealing with functions with discrete parameters,or particularly erratic target space it might be beneficial to increase the value of the `alpha` parameter. This parameters controls how much noise the GP can handle, so increase it whenever you think that extra flexibility is needed.
%% Cell type:markdown id: tags:
### 3.3 Changing kernels
By default this package uses the Matern 2.5 kernel. Depending on your use case you may find that tunning the GP kernel could be beneficial. You're on your own here since these are very specific solutions to very specific problems.
By default this package uses the Matern 2.5 kernel. Depending on your use case you may find that tunning the GP kernel could be beneficial. You're on your own here since these are very specific solutions to very specific problems. You should start with the [scikit learn docs](https://scikit-learn.org/stable/modules/gaussian_process.html#kernels-for-gaussian-processes)
%% Cell type:markdown id: tags:
## Observers Continued
Observers are objects that subscribe and listen to particular events fired by the `BayesianOptimization` object.
When an event gets fired a callback function is called with the event and the `BayesianOptimization` instance passed as parameters. The callback can be specified at the time of subscription. If none is given it will look for an `update` method from the observer.
%% Cell type:code id: tags:
``` python
from bayes_opt.event import DEFAULT_EVENTS, Events
```
%% Cell type:code id: tags:
``` python
optimizer = BayesianOptimization(
f=black_box_function,
pbounds={'x': (-2, 2), 'y': (-3, 3)},
verbose=2,
random_state=1,
)
```
%% Cell type:code id: tags:
``` python
class BasicObserver:
def update(self, event, instance):
"""Does whatever you want with the event and `BayesianOptimization` instance."""
print("Event `{}` was observed".format(event))
```
%% Cell type:code id: tags:
``` python
my_observer = BasicObserver()
optimizer.subscribe(
event=Events.OPTIMIZATION_STEP,
subscriber=my_observer,
callback=None, # Will use the `update` method as callback
)
```
%% Cell type:markdown id: tags:
Alternatively you have the option to pass a completely different callback.
%% Cell type:code id: tags:
``` python
def my_callback(event, instance):
print("Go nuts here!")
optimizer.subscribe(
event=Events.OPTIMIZATION_START,
subscriber="Any hashable object",
callback=my_callback,
)
```
%% Cell type:code id: tags:
``` python
optimizer.maximize(init_points=1, n_iter=2)
```
%% Output
Go nuts here!
Event `optimization:step` was observed
Event `optimization:step` was observed
Event `optimization:step` was observed
%% Cell type:markdown id: tags:
For a list of all default events you can checkout `DEFAULT_EVENTS`
%% Cell type:code id: tags:
``` python
DEFAULT_EVENTS
```
%% Output
['optimization:start', 'optimization:step', 'optimization:end']
......
examples/domain_reduction.ipynb
+ 3
− 3
View file @ e0854566
%% Cell type:markdown id: tags:
# Sequential Domain Reduction
## Background
Sequential domain reduction is a process where the bounds of the optimization problem are mutated (typically contracted) to reduce the time required to converge to an optimal value. The advantage of this method is typically seen when a cost function is particularly expensive to calculate, or if the optimization routine oscillates heavily.
## Basics
The basic steps are a *pan* and a *zoom*. These two steps are applied at one time, therefore updating the problem search space every iteration.
**Pan**: recentering the region of interest around the most optimal point found.
**Zoom**: contract the region of interest.
![](../static/sdr.png)
![](../docsrc/static/sdr.png)
## Parameters
There are three parameters for the built-in `SequentialDomainReductionTransformer` object:
$\gamma_{osc}:$ shrinkage parameter for oscillation. Typically [0.5-0.7]. Default = 0.7
$\gamma_{pan}:$ panning parameter. Typically 1.0. Default = 1.0
$\eta:$ zoom parameter. Default = 0.9
More information can be found in this reference document:
---
Title: "On the robustness of a simple domain reduction scheme for simulation‐based optimization"
Date: 2002
Author: Stander, N. and Craig, K.
%% Cell type:markdown id: tags:
---
---
Let's start by importing the packages we'll be needing
%% Cell type:code id: tags:
``` python
import numpy as np
from bayes_opt import BayesianOptimization
from bayes_opt import SequentialDomainReductionTransformer
import matplotlib.pyplot as plt
```
%% Cell type:markdown id: tags:
Now let's create an example cost function. This is the [Ackley function](https://en.wikipedia.org/wiki/Ackley_function), which is quite non-linear.
%% Cell type:code id: tags:
``` python
def ackley(**kwargs):
x = np.fromiter(kwargs.values(), dtype=float)
arg1 = -0.2 * np.sqrt(0.5 * (x[0] ** 2 + x[1] ** 2))
arg2 = 0.5 * (np.cos(2. * np.pi * x[0]) + np.cos(2. * np.pi * x[1]))
return -1.0 * (-20. * np.exp(arg1) - np.exp(arg2) + 20. + np.e)
```
%% Cell type:markdown id: tags:
We will use the standard bounds for this problem.
%% Cell type:code id: tags:
``` python
pbounds = {'x': (-5, 5), 'y': (-5, 5)}
```
%% Cell type:markdown id: tags:
This is where we define our `bound_transformer` , the Sequential Domain Reduction Transformer
%% Cell type:code id: tags:
``` python
bounds_transformer = SequentialDomainReductionTransformer(minimum_window=0.5)
```
%% Cell type:markdown id: tags:
Now we can set up two identical optimization problems, except one has the `bound_transformer` variable set.
%% Cell type:code id: tags:
``` python
mutating_optimizer = BayesianOptimization(
f=ackley,
pbounds=pbounds,
verbose=0,
random_state=1,
bounds_transformer=bounds_transformer
)
```
%% Cell type:code id: tags:
``` python
mutating_optimizer.maximize(
init_points=2,
n_iter=50,
)
```
%% Cell type:code id: tags:
``` python
standard_optimizer = BayesianOptimization(
f=ackley,
pbounds=pbounds,
verbose=0,
random_state=1,
)
```
%% Cell type:code id: tags:
``` python
standard_optimizer.maximize(
init_points=2,
n_iter=50,
)
```
%% Cell type:markdown id: tags:
After both have completed we can plot to see how the objectives performed. It's quite obvious to see that the Sequential Domain Reduction technique contracted onto the optimal point relatively quick.
%% Cell type:code id: tags:
``` python
plt.plot(mutating_optimizer.space.target, label='Mutated Optimizer')
plt.plot(standard_optimizer.space.target, label='Standard Optimizer')
plt.legend()
```
%% Output
<matplotlib.legend.Legend at 0x23439267280>
%% Cell type:markdown id: tags:
Now let's plot the actual contraction of one of the variables (`x`)
%% Cell type:code id: tags:
``` python
# example x-bound shrinking - we need to shift the x-axis by the init_points as the bounds
# transformer only mutates when searching - not in the initial phase.
x_min_bound = [b[0][0] for b in bounds_transformer.bounds]
x_max_bound = [b[0][1] for b in bounds_transformer.bounds]
x = [x[0] for x in mutating_optimizer.space.params]
bounds_transformers_iteration = list(range(2, len(x)))
```
%% Cell type:code id: tags:
``` python
plt.plot(bounds_transformers_iteration, x_min_bound[1:], label='x lower bound')
plt.plot(bounds_transformers_iteration, x_max_bound[1:], label='x upper bound')
plt.plot(x[1:], label='x')
plt.legend()
```
%% Output
<matplotlib.legend.Legend at 0x23439267fd0>
......
pyproject.toml
+ 6
− 2
View file @ e0854566
[tool.poetry]
name = "bayesian-optimization"
version = "1.5.0"
version = "1.5.1"
description = "Bayesian Optimization package"
authors = ["Fernando Nogueira"]
license = "MIT"
......@@ -10,7 +10,7 @@ packages = [{include = "bayes_opt"}]
[tool.poetry.dependencies]
python = "^3.9"
scikit-learn = "^1.0.0"
numpy = "^1.9.0"
numpy = ">=1.25"
scipy = "^1.0.0"
colorama = "^0.4.6"
......@@ -31,6 +31,10 @@ nbformat = "^5.9.2"
nbconvert = "^7.14.2"
jupyter = "^1.0.0"
matplotlib = "^3.0"
sphinx = "^7.3.7"
nbsphinx = "^0.9.4"
sphinx-rtd-theme = "^2.0.0"
myst-parser = "^3.0.1"
[build-system]
......
tests/test_logs_multiple_constraints.log 0 → 100644
+ 12
− 0
View file @ e0854566
{"target": -34660.799399896976, "constraint": [0.2708972584125664, 0.42357209835559106], "params": {"x": -50.183952461055, "y": 180.28572256396646}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:44", "elapsed": 0.0, "delta": 0.0}}
{"target": -10089.822899242456, "constraint": [-0.9510813312336377, 0.9973445494788389], "params": {"x": 92.79757672456203, "y": 39.46339367881464}, "allowed": false, "datetime": {"datetime": "2024-06-20 15:45:44", "elapsed": 0.000321, "delta": 0.000321}}
{"target": -10506.089080732436, "constraint": [0.6131701761022414, -0.9564999545661308], "params": {"x": 95.21519271989214, "y": 38.96256255900324}, "allowed": false, "datetime": {"datetime": "2024-06-20 15:45:44", "elapsed": 0.056496, "delta": 0.056175}}
{"target": -997.8139138494062, "constraint": [-0.9916913121401841, -0.6495843128640307], "params": {"x": 18.482175449759072, "y": -24.636362934194256}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:44", "elapsed": 0.556892, "delta": 0.500396}}
{"target": -19477.39999548628, "constraint": [0.4566191751114183, 0.10531295488159137], "params": {"x": -30.134980186677655, "y": -135.2728254812192}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:45", "elapsed": 1.025239, "delta": 0.468347}}
{"target": -2557.6177973621116, "constraint": [0.9347247196809806, 0.9180512307035334], "params": {"x": -49.8799963378478, "y": 9.402604518743091}, "allowed": false, "datetime": {"datetime": "2024-06-20 15:45:45", "elapsed": 1.359698, "delta": 0.334459}}
{"target": -39604.24318900592, "constraint": [0.8549175086147126, 0.9001717563474866], "params": {"x": -197.96772313017533, "y": -19.347574490837093}, "allowed": false, "datetime": {"datetime": "2024-06-20 15:45:45", "elapsed": 1.595204, "delta": 0.235506}}
{"target": -80400.0, "constraint": [-0.9999999999999999, 0.5252963386425359], "params": {"x": 200.0, "y": -200.0}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:45", "elapsed": 1.788137, "delta": 0.192933}}
{"target": -79600.0, "constraint": [0.5252963386425359, -0.9999999999999999], "params": {"x": 200.0, "y": 200.0}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:46", "elapsed": 2.146403, "delta": 0.358266}}
{"target": -80400.0, "constraint": [0.5252963386425359, -0.9999999999999999], "params": {"x": -200.0, "y": -200.0}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:46", "elapsed": 2.518581, "delta": 0.372178}}
{"target": -79600.0, "constraint": [-0.9999999999999999, 0.5252963386425359], "params": {"x": -200.0, "y": 200.0}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:47", "elapsed": 3.028643, "delta": 0.510062}}
{"target": -4290.396951749328, "constraint": [-0.32857141283617775, -0.3643611387948609], "params": {"x": -8.207850284687554, "y": 65.99252376584158}, "allowed": true, "datetime": {"datetime": "2024-06-20 15:45:47", "elapsed": 3.494066, "delta": 0.465423}}
tests/test_util.py
+ 34
− 0
View file @ e0854566
......@@ -189,6 +189,40 @@ def test_logs_constraint():
assert len(optimizer.space) == 7
def test_logs_constraint():
def f(x, y):
return -x ** 2 - (y - 1) ** 2 + 1
def c(x, y):
return np.array([
- np.cos(x) * np.cos(y) + np.sin(x) * np.sin(y),
- np.cos(x) * np.cos(-y) + np.sin(x) * np.sin(-y)
])
constraint_lower = np.array([-np.inf, -np.inf])
constraint_upper = np.array([0.6, 0.6])
constraint = NonlinearConstraint(c, constraint_lower, constraint_upper)
optimizer = BayesianOptimization(
f=f,
pbounds={"x": (-200, 200), "y": (-200, 200)},
constraint=constraint
)
with pytest.raises(KeyError):
load_logs(optimizer, [str(test_dir / "test_logs.log")])
with pytest.raises(ValueError):
load_logs(optimizer, [str(test_dir / "test_logs_constrained.log")])
load_logs(optimizer, [str(test_dir / "test_logs_multiple_constraints.log")])
print(optimizer.space)
assert len(optimizer.space) == 12
if __name__ == '__main__':
r"""
CommandLine:
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment