Skip to content

Add comments on the reproducibility of HyperBandPruner#6018

Merged
gen740 merged 2 commits intooptuna:masterfrom
nzw0301:hyperband-reprodicible-note
Mar 28, 2025
Merged

Add comments on the reproducibility of HyperBandPruner#6018
gen740 merged 2 commits intooptuna:masterfrom
nzw0301:hyperband-reprodicible-note

Conversation

@nzw0301
Copy link
Copy Markdown
Member

@nzw0301 nzw0301 commented Mar 23, 2025

Motivation

HyperBand Pruner has not been reproducible when study_name is None in create_study method, see the example code in #3893. This question is still FAQ, presumably, ex: #6017.

A long time ago, I'd added the note by #3901. However, this note was removed in #4131 nicely to address the hash function issue for parallel optimisation and its followup #5075, probably mistakenly.

Description of the changes

Re-introduce a few notes on the HyperBand's reproducibly.

@nzw0301 nzw0301 added the document Documentation related. label Mar 23, 2025
@nzw0301 nzw0301 changed the title Add the reproducibility note of HyperBandPruner Add comments on the reproducibility of HyperBandPruner Mar 23, 2025
@HideakiImamura
Copy link
Copy Markdown
Member

@gen740 @sawa3030 Could you review this PR?

Copy link
Copy Markdown
Collaborator

@sawa3030 sawa3030 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I've confirmed that the pruner is not reproducible when study_name is not set. However, setting study_name ensures reproducibility. I verified this using the code provided in #3893.

Copy link
Copy Markdown
Member

@gen740 gen740 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comment.

Co-authored-by: Gen <54583542+gen740@users.noreply.github.com>
Copy link
Copy Markdown
Member

@gen740 gen740 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gen740 gen740 added this to the v4.3.0 milestone Mar 28, 2025
@gen740 gen740 merged commit b697e70 into optuna:master Mar 28, 2025
14 checks passed
@nzw0301 nzw0301 deleted the hyperband-reprodicible-note branch March 28, 2025 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

document Documentation related.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants