Skip to content

Allow fail_state_trials show warning when heartbeat is enabled#3301

Merged
HideakiImamura merged 2 commits intooptuna:masterfrom
himkt:hotfix/catch-warnings
Feb 9, 2022
Merged

Allow fail_state_trials show warning when heartbeat is enabled#3301
HideakiImamura merged 2 commits intooptuna:masterfrom
himkt:hotfix/catch-warnings

Conversation

@himkt
Copy link
Copy Markdown
Member

@himkt himkt commented Feb 9, 2022 •
edited
Loading

In the current latest alpha release, the same warning message is displayed repeatedly, which is not intentional.
This behavior seemingly comes from the known issue of warnings.catch_warnings, which was reported in https://bugs.python.org/issue29672.

Invoking warnings.catch_warnings has a side-effect, and it would be better not to invoke it for not only Optuna but also the library users. So, in this PR, I remove catch_warnings in _run_trial and check if heartbeat is enabled by the user instead.

Motivation

Stop showing a warning repeatedly.

Description of the changes

  • ecb80fb remove catch_warnings and check if heartbeat is enabled
  • 9e2bea9 flake8 fix (remove unused import)
import optuna


def objective(trial):
    return trial.suggest_float("x", 0, 1)


study = optuna.create_study()
study.optimize(objective, n_trials=5)

Before

> python tmp/optimize_test.py                                                                                                                                                                                                          2022-02-09 12:11:42
[I 2022-02-09 12:11:45,937] A new study created in memory with name: no-name-47406b8b-8af8-40b1-9c90-d9cb1e2a8d42
/Users/himkt/work/github.com/himkt/optuna/optuna/trial/_trial.py:169: FutureWarning: UniformDistribution has been deprecated in v3.0.0. This feature will be removed in v6.0.0. See https://github.com/optuna/optuna/releases/tag/v3.0.0. Use :class:`~optuna.distributions.FloatDistribution` instead.
  distribution = UniformDistribution(low=low, high=high)
[I 2022-02-09 12:11:45,939] Trial 0 finished with value: 0.7981646615533516 and parameters: {'x': 0.7981646615533516}. Best is trial 0 with value: 0.7981646615533516.
/Users/himkt/work/github.com/himkt/optuna/optuna/trial/_trial.py:169: FutureWarning: UniformDistribution has been deprecated in v3.0.0. This feature will be removed in v6.0.0. See https://github.com/optuna/optuna/releases/tag/v3.0.0. Use :class:`~optuna.distributions.FloatDistribution` instead.
  distribution = UniformDistribution(low=low, high=high)
[I 2022-02-09 12:11:45,939] Trial 1 finished with value: 0.7292757393008622 and parameters: {'x': 0.7292757393008622}. Best is trial 1 with value: 0.7292757393008622.
/Users/himkt/work/github.com/himkt/optuna/optuna/trial/_trial.py:169: FutureWarning: UniformDistribution has been deprecated in v3.0.0. This feature will be removed in v6.0.0. See https://github.com/optuna/optuna/releases/tag/v3.0.0. Use :class:`~optuna.distributions.FloatDistribution` instead.
  distribution = UniformDistribution(low=low, high=high)
[I 2022-02-09 12:11:45,959] Trial 2 finished with value: 0.4639811119742068 and parameters: {'x': 0.4639811119742068}. Best is trial 2 with value: 0.4639811119742068.
/Users/himkt/work/github.com/himkt/optuna/optuna/trial/_trial.py:169: FutureWarning: UniformDistribution has been deprecated in v3.0.0. This feature will be removed in v6.0.0. See https://github.com/optuna/optuna/releases/tag/v3.0.0. Use :class:`~optuna.distributions.FloatDistribution` instead.
  distribution = UniformDistribution(low=low, high=high)
[I 2022-02-09 12:11:45,974] Trial 3 finished with value: 0.9529777145282745 and parameters: {'x': 0.9529777145282745}. Best is trial 2 with value: 0.4639811119742068.
/Users/himkt/work/github.com/himkt/optuna/optuna/trial/_trial.py:169: FutureWarning: UniformDistribution has been deprecated in v3.0.0. This feature will be removed in v6.0.0. See https://github.com/optuna/optuna/releases/tag/v3.0.0. Use :class:`~optuna.distributions.FloatDistribution` instead.
  distribution = UniformDistribution(low=low, high=high)
[I 2022-02-09 12:11:45,975] Trial 4 finished with value: 0.5049101745390385 and parameters: {'x': 0.5049101745390385}. Best is trial 2 with value: 0.4639811119742068.

After

> python tmp/optimize_test.py                                                                                                                                                                                                          2022-02-09 12:11:35
[I 2022-02-09 12:11:39,216] A new study created in memory with name: no-name-76610cf6-85f2-42da-aabb-51442d6e6c4e
/Users/himkt/work/github.com/himkt/optuna/optuna/trial/_trial.py:169: FutureWarning: UniformDistribution has been deprecated in v3.0.0. This feature will be removed in v6.0.0. See https://github.com/optuna/optuna/releases/tag/v3.0.0. Use :class:`~optuna.distributions.FloatDistribution` instead.
  distribution = UniformDistribution(low=low, high=high)
[I 2022-02-09 12:11:39,221] Trial 0 finished with value: 0.12598191592979535 and parameters: {'x': 0.12598191592979535}. Best is trial 0 with value: 0.12598191592979535.
[I 2022-02-09 12:11:39,222] Trial 1 finished with value: 0.5999996391722033 and parameters: {'x': 0.5999996391722033}. Best is trial 0 with value: 0.12598191592979535.
[I 2022-02-09 12:11:39,223] Trial 2 finished with value: 0.9303211620451699 and parameters: {'x': 0.9303211620451699}. Best is trial 0 with value: 0.12598191592979535.
[I 2022-02-09 12:11:39,224] Trial 3 finished with value: 0.9111266953883 and parameters: {'x': 0.9111266953883}. Best is trial 0 with value: 0.12598191592979535.
[I 2022-02-09 12:11:39,224] Trial 4 finished with value: 0.9499001545534816 and parameters: {'x': 0.9499001545534816}. Best is trial 0 with value: 0.12598191592979535.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 9, 2022

Codecov Report

Merging #3301 (9e2bea9) into master (9595571) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3301      +/-   ##
==========================================
- Coverage   91.76%   91.73%   -0.03%     
==========================================
  Files         146      146              
  Lines       12091    12089       -2     
==========================================
- Hits        11095    11090       -5     
- Misses        996      999       +3
Impacted Files Coverage Δ
optuna/study/_optimize.py 98.30% <100.00%> (-0.02%) ⬇️
optuna/storages/_heartbeat.py 92.85% <0.00%> (-7.15%) ⬇️
optuna/integration/botorch.py 97.80% <0.00%> (-0.88%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9595571...9e2bea9. Read the comment docs.

@himkt
Copy link
Copy Markdown
Member Author

himkt commented Feb 9, 2022

@HideakiImamura PTAL. cc. @hvy

hvy
hvy approved these changes Feb 9, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@hvy hvy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@hvy hvy self-assigned this Feb 9, 2022
HideakiImamura
HideakiImamura approved these changes Feb 9, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@HideakiImamura HideakiImamura left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@HideakiImamura HideakiImamura merged commit 002d1e1 into optuna:master Feb 9, 2022
@HideakiImamura HideakiImamura added this to the v3.0.0-b0 milestone Feb 9, 2022
@HideakiImamura HideakiImamura added the bug Issue/PR about behavior that is broken. Not for typos/examples/CI/test but for Optuna itself. label Feb 9, 2022
HideakiImamura added a commit to HideakiImamura/optuna that referenced this pull request Feb 9, 2022
@HideakiImamura
Merge pull request optuna#3301 from himkt/hotfix/catch-warnings
9a0d937 
Allow `fail_state_trials` show warning when heartbeat is enabled
@HideakiImamura HideakiImamura mentioned this pull request Feb 9, 2022
Merged
@himkt himkt deleted the hotfix/catch-warnings branch February 9, 2022 05:09
@himkt himkt mentioned this pull request Feb 10, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hvy hvy hvy approved these changes

@HideakiImamura HideakiImamura HideakiImamura approved these changes

Assignees

@hvy hvy

@HideakiImamura HideakiImamura

Labels

bug Issue/PR about behavior that is broken. Not for typos/examples/CI/test but for Optuna itself.

Projects

None yet

Milestone

v3.0.0-b0

Development

Successfully merging this pull request may close these issues.

4 participants

@himkt @codecov-commenter @hvy @HideakiImamura