acerbilab / pybads Goto Github PK

In example notebooks, the links to full code like pybads_example_1_basic_usage.py are not correct. The py scripts are now in tests/bads/examples/.

Maybe it's better to place these examples scripts to ./examples/ or ./examples/scripts? Since they are not actually part of tests.

My objective function can accept vectors of different sizes. Using a size of 60:

Initial starting point is invalid or not provided. Initial point randomly sampled uniformly from plausible box

Beginning optimization of a DETERMINISTIC objective function

 Iteration    f-count         f(x)           MeshScale          Method             Actions
     0           2         211.912               1                                 Uncertainty test
     0          66         109.632               1         Initial mesh            Initial points
     0          67         103.554               1       Successful poll           Train
Traceback (most recent call last):
  File "c:\Users\Mokdad\Desktop\Code\Optm.py", line 191, in <module>
    result2=bads.optimize()
  File "C:\Users\Mokdad\AppData\Roaming\Python\Python39\site-packages\pybads\bads\bads.py", line 1233, in optimize
    ) = self._search_step_(gp)
  File "C:\Users\Mokdad\AppData\Roaming\Python\Python39\site-packages\pybads\bads\bads.py", line 1580, in _search_step_
    u_search_set, z = self.search_es_hedge(
  File "C:\Users\Mokdad\AppData\Roaming\Python\Python39\site-packages\pybads\search\search_hedge.py", line 78, in __call__
    us, z = search(
  File "C:\Users\Mokdad\AppData\Roaming\Python\Python39\site-packages\pybads\search\es_search.py", line 118, in __call__
    self.sqrt_sigma = self._initialize_(u, gp, optim_state, sum_rule)
  File "C:\Users\Mokdad\AppData\Roaming\Python\Python39\site-packages\pybads\search\es_search.py", line 239, in _initialize_
    C = ucov(
  File "C:\Users\Mokdad\AppData\Roaming\Python\Python39\site-packages\pybads\search\es_search.py", line 313, in ucov
    weights = w.reshape(
ValueError: maximum supported dimension for an ndarray is 32, found 61

Using a length of 32 works, anything greater does not. This is due to the limitation in numpy array dimension not exceeding 32.

I left a comment on your JOSS submission.

• This paper mentioned that PyBADS or BADS is out of the Bayesian optimization field. For example, BayesOpt requires specific technical knowledge to be implemented or tuned beyond simple tasks. I think that it is only applied for the vanilla Bayesian optimization, and moreover BADS, which belongs to one of Bayesian optimization algorithms, tried to improve the vanilla Bayesian optimization, which implies that BADS is on the line of research on Bayesian optimization.
• Do you have any references or evidences for the function evaluation requires more than 0.1 seconds (BADS) and evaluation costs of hours or more (Bayesian optimization)? In my experience, that is not true at least for Bayesian optimization.
• Do you think if any acquisition functions like EI or PI can be used in PyBADS instead of LCB?
• I think that this project heavily relies on BADS. If I am not wrong, this is the re-implementation of BADS. The authors need to discuss this more thoroughly in the paper.
• I think this issue is the most serious one. the Method section is copied and pasted from the BADS repository (the same sentences are in this repository, though). For example,

In the poll stage, points are evaluated on a mesh by taking steps in one direction at a time, until an improvement is found or all directions have been tried. The step size is doubled in case of success, halved otherwise. in the BADS repository,

In the poll stage, points are evaluated on a mesh by taking steps in one (non-orthogonal) direction at a time, until an improvement is found or all directions have been tried. The step size is doubled in case of success, halved otherwise. in your paper.

Also,

In the search stage, a Gaussian process (GP) is fit to a (local) subset of the points evaluated so far. Then, we iteratively choose points to evaluate according to a lower confidence bound strategy that trades off between exploration of uncertain regions (high GP uncertainty) and exploitation of promising solutions (low GP mean). in the BADS repository,

In the search stage, a Gaussian process (GP) surrogate model (Rasmussen & Williams, 2006) of the target function is fit to a local subset of the points evaluated so far. New points to evaluate are quickly chosen according to a lower confidence bound strategy that trades off between exploration of uncertain regions (high GP uncertainty) and exploitation of promising solutions (low GP mean) in your paper.

I think that this should be fixed, even if part of the list of the authors is overlapped.

Please take a look at my comment and potentially update your paper.

I think you need to cite this SciPy paper.

You can cite this paper on the quasi-Monte Carlo method you used.

Here is the code snippet that reproduces the issue:

    def foo(x):
        return x[0]**2

    x0 = np.array([1])
    lower_bounds = np.array([-10])
    upper_bounds = np.array([10])
    plausible_lower_bounds = np.array([-5])
    plausible_upper_bounds = np.array([5])
    bads = BADS(foo, x0, lower_bounds, upper_bounds, plausible_lower_bounds, plausible_upper_bounds)

Traceback (most recent call last):
  File "D:\Miniconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3505, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-41-0b496bbed19e>", line 9, in <module>
    bads = BADS(foo, x0, lower_bounds, upper_bounds, plausible_lower_bounds, plausible_upper_bounds)
  File "D:\Miniconda3\lib\site-packages\pybads\bads\bads.py", line 245, in __init__
    self.optim_state = self._init_optim_state_()
  File "D:\Miniconda3\lib\site-packages\pybads\bads\bads.py", line 602, in _init_optim_state_
    self.var_transf = VariableTransformer(
  File "D:\Miniconda3\lib\site-packages\pybads\variable_transformer\variables_transformer.py", line 117, in __init__
    ) = self.__create_hypercube_trans__()
  File "D:\Miniconda3\lib\site-packages\pybads\variable_transformer\variables_transformer.py", line 164, in __create_hypercube_trans__
    and (self.pub[:, i] / self.plb[:, i] >= 10).item()
ValueError: can only convert an array of size 1 to a Python scalar

Could you list up the Python versions supported in the README or anywhere in the repository?

Thank you for sharing your valuable project. I appreciate your endeavor to develop this.

Generally, this repository is well-developed and well-documented. Also, proper and well-established examples are provided.

I intentionally left the separate issues:

• #43
• #44
• #45
• #48
• #46

Please close them if they are resolved.

If done or answered, I will update my checklist.

Best regards,
Jungtaek.

Hi, I'm testing your package and it seems to be working amazingly for my use case :)

However, I have two questions regarding how BADS can be initialized:

1. If a previous BADS run has stopped for some reason, is there a way to continue it from that point?
2. More generally, if I have a known set of points of the function to optimize, is there some way I can feed them into BADS, maybe as initial points?

Thanks!

My objective function requires additional parameters and data to be passed to it. The MATLAB implementation of BADS allows you to pass this data as the last argument when running BADS. Would a similar functionality be made available for pyBADS, where I can pass in the additional arguments when initializing BADS?

I think that you wanted to follow the style of the SciPy's OptimizeResult.

If needed, you can mention that it is a similar feature of the OptimizeResult in SciPy.

It will help users understand how they would use this feature.

It seems a bit tedious to import this way. Is from pybads import BADS better?

Hi!
first, congrats on porting this from Matlab!
I am just wondering how can one pass custom data to the objective function. In the docs, I don't seem to be able to find an option for that. Thank you!