Skip to content

Mention reproducibility of HyperBandPruner#3901

Merged
contramundum53 merged 1 commit intooptuna:masterfrom
nzw0301:mention-hyper-band-reproducibility
Aug 19, 2022
Merged

Mention reproducibility of HyperBandPruner#3901
contramundum53 merged 1 commit intooptuna:masterfrom
nzw0301:mention-hyper-band-reproducibility

Conversation

@nzw0301
Copy link
Copy Markdown
Member

@nzw0301 nzw0301 commented Aug 18, 2022

Motivation

HyperbandPruner has randomness depending on study_name and trial.number by using the hash function as defined by

optuna/optuna/pruners/_hyperband.py

Lines 247 to 249 in 0dd1e0e

n = (
hash("{}_{}".format(study.study_name, trial.number))
% self._total_trial_allocation_budget
. It would be great to explain how to get reproducible pruning results in Hyperband's page and FAQ's reproducibility section.

Description of the changes

  1. Add a new seealso section to HyperBandPruner page
  2. Mention HyperBandPruner's randomness in FAQ section on reproducibility
  3. (a minor one) Update the sampler part too in the FAQ section since another sampler such as NSGAIISampler also takes seed argument to specify the random seed.

@nzw0301 nzw0301 added the document Documentation related. label Aug 18, 2022
@github-actions github-actions bot added the optuna.pruners Related to the `optuna.pruners` submodule. This is automatically labeled by github-actions. label Aug 18, 2022
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Aug 18, 2022

Codecov Report

Merging #3901 (21d5895) into master (0dd1e0e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3901   +/-   ##
=======================================
  Coverage   90.57%   90.57%           
=======================================
  Files         164      164           
  Lines       12601    12601           
=======================================
  Hits        11413    11413           
  Misses       1188     1188
Impacted Files Coverage Δ
optuna/pruners/_hyperband.py 98.80% <ø> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

contramundum53
contramundum53 approved these changes Aug 19, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@contramundum53 contramundum53 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the fix!

@contramundum53 contramundum53 added this to the v3.0.0 milestone Aug 19, 2022
@contramundum53 contramundum53 merged commit 5a1bb75 into optuna:master Aug 19, 2022
@nzw0301 nzw0301 deleted the mention-hyper-band-reproducibility branch February 24, 2024 04:36
@nzw0301 nzw0301 mentioned this pull request Mar 23, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@contramundum53 contramundum53 contramundum53 approved these changes

Assignees

No one assigned

Labels

document Documentation related. optuna.pruners Related to the `optuna.pruners` submodule. This is automatically labeled by github-actions.

Projects

None yet

Milestone

v3.0.0

Development

Successfully merging this pull request may close these issues.

3 participants

@nzw0301 @codecov-commenter @contramundum53