Stop using hash function in _get_bracket_id in HyperbandPruner#4131
Conversation
|
I am concerned that if n_trials is not sufficiently larger than total_trial_allocation_budget, the bracket_id will be biased towards smaller ones by this change. This happens when max_resource is large and n_trials is small, so I wonder if it would make a difference in results in NN hyperparameter optimization. |
Codecov Report
@@ Coverage Diff @@
## master #4131 +/- ##
==========================================
- Coverage 90.10% 90.09% -0.02%
==========================================
Files 161 161
Lines 12655 12656 +1
==========================================
- Hits 11403 11402 -1
- Misses 1252 1254 +2
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
To avoid the bias discussed above, I modified the implementation to use crc32, which was mentioned in #3083. The result of the above code after modification is |
not522
left a comment
There was a problem hiding this comment.
Could you remove this note?
optuna/optuna/pruners/_hyperband.py
Lines 63 to 68 in f60976d
|
Thanks for the review. I simply deleted the note, is this ok? |
_get_bracket_id in HyperbandPruner
Motivation
Fix the bug reported in #3083 that _get_bracket_id of Hyperband returns different results depending on the process.
Description of the changes
Simply stopped using
hashas suggested in #809 (comment).