Cache split in TPE for high-dimensional optimization#5464
Cache split in TPE for high-dimensional optimization#5464nabenabe0928 wants to merge 11 commits intooptuna:masterfrom
Conversation
|
@not522 |
optuna/_hypervolume/utils.py
Outdated
| @@ -1,3 +1,5 @@ | |||
| from __future__ import annotations | |||
There was a problem hiding this comment.
I believe this change is not relevant to this PR.
This change would be good to include when using features from __future__.
|
Sorry, I am temporarily busy because my primary computer is not working, and I suppose @HideakiImamura -san is the appropriate reviewer. @HideakiImamura could you review this PR? |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5464 +/- ##
==========================================
+ Coverage 89.52% 89.74% +0.22%
==========================================
Files 194 195 +1
Lines 12626 12592 -34
==========================================
- Hits 11303 11301 -2
+ Misses 1323 1291 -32 ☔ View full report in Codecov by Sentry. |
|
We discussed internally and decided to close this PR. |
|
Just for future reminder, I will leave some comments: This PR does not cause any issues between processesAs each trial is sampled in a thread and a sampling of a specific trial will not be scattered on multiple processes or threads, we do not have to be concerned about the missing cached data in another process. However, when using multiple threads, we need to take care of another thread overwriting the cache before one trial completes its sampling. Anyways, missing cache does not cause any issue because we simply need to re-calculate the split, which incurs some more computational burden. |
Motivation
As TPE significantly slows down for high-dimensional optimization, this PR introduces caching mechanism for the TPE split.
Description of the changes
For
n_trials=1000anddim=10, the runtimes are the following: