Safe CMA-ES (GECCO 2024) #187
Conversation
test_safecma
|
@c-bata |
| @@ -463,3 +463,4 @@ Rate Adaptation: Can CMA-ES with Default Population Size Solve Multimodal | |||
| and Noisy Problems?, GECCO, 2023.](https://arxiv.org/abs/2304.03473) | |||
| * [Nomura and Shibata 2024] [M. Nomura, M. Shibata, cmaes : A Simple yet Practical Python Library for CMA-ES, arXiv:2402.01373, 2024.](https://arxiv.org/abs/2402.01373) | |||
| * [Ros and Hansen 2008] [R. Ros, N. Hansen, A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, PPSN, 2008.](https://hal.inria.fr/inria-00287367/document) | |||
| * [Uchida et al. 2024] [K. Uchida, R. Hamano, M. Nomura, S. Saito, S. Shirakawa, CMA-ES for Safe Optimization, GECCO, 2024.](https://arxiv.org/abs/2405.10534) | |||
There was a problem hiding this comment.
Could you provide a brief description and usage for SafeCMA (like LRA-CMA-ES)?
There was a problem hiding this comment.
It would be clearer to explicitly state that the current implementation assumes no noise.
|
|
||
| cov: | ||
| A covariance matrix (optional). | ||
| """ |
There was a problem hiding this comment.
Please add the paper link like this.
| assert c1 <= 1 - cmu, "invalid learning rate for the rank-one update" | ||
| assert cmu <= 1 - c1, "invalid learning rate for the rank-μ update" | ||
|
|
||
| cm = 1 # (eq. 54) |
There was a problem hiding this comment.
I suggest removing comments like eq. ~ (for Hansen's Tutorial paper) in this PR, as they could be confusing when compared with the equations in the SafeCMA paper.
There was a problem hiding this comment.
On the other hand, adding equation numbers for SafeCMA-specific operations would improve clarity.
| likelihood = gpytorch.likelihoods.GaussianLikelihood( | ||
| noise_constraint=gpytorch.constraints.GreaterThan(0) | ||
| ) | ||
| likelihood.noise = 0 |
There was a problem hiding this comment.
I'm curious about the validity of this setting because, as mentioned in this thread:
Some small value, but don't make it too small or numerical performance will suffer. I recommend 1e-4.
cornellius-gp/gpytorch#1196 (comment)
This implies there's room for improvement in this setting, but I'm not sure for now, so let's see how it goes.
| if out_scalar: | ||
| grad_norm = grad_norm.mean().to(torch.float64) | ||
|
|
||
| return -grad_norm |
There was a problem hiding this comment.
df is defined as # function that returns the norm of gradient, but it currently returns a negative value, which seems odd.
I understand this is intended for scipy.optimize.minimize, but it would be better to align it with the description.
| safe_seeds = (np.random.rand(safe_seeds_num, dim) * 2 - 1) * 5 | ||
| safe_seeds[:, 0] = - np.abs(safe_seeds[:, 0]) | ||
|
|
||
| # evaluation of safe seeds (with multiple safety functions) |
There was a problem hiding this comment.
with multiple safety functions
with a single safety function?
| safe_seeds = (np.random.rand(safe_seeds_num, dim) * 2 - 1) * 5 | ||
| safe_seeds[:, 0] = -np.abs(safe_seeds[:, 0]) | ||
|
|
||
| # evaluation of safe seeds (with multiple safety functions) |
There was a problem hiding this comment.
with multiple safety functions
with a single safety function?
|
@kento031 |
| try: | ||
| from ._safe_cma import SafeCMA # NOQA | ||
| except ImportError: | ||
| pass # Implementation of Safe CMA-ES requires scipy, gpytorch, and torch | ||
|
|
There was a problem hiding this comment.
It seems that the current approach is somewhat tricky, as users may be confused if cmaes.SafeCMA doesn’t exist in certain Python environments.
How about simply removing SafeCMA from __init__.py and renaming cmaes/_safe_cma.py to cmaes/safe_cma.py?
| try: | |
| from ._safe_cma import SafeCMA # NOQA | |
| except ImportError: | |
| pass # Implementation of Safe CMA-ES requires scipy, gpytorch, and torch |
There was a problem hiding this comment.
Thanks for the great suggestion! It looks good to me.
@kento031
Could you make this modification if it’s okay?
|
@nomuramasahir0 @c-bata |
This is the implementation of Safe CMA-ES [Uchida+, GECCO 2024].
https://arxiv.org/abs/2405.10534