Study.optimize cannot be stopped properly when sampler=GridSampler and pruner=HyperbandPruner #2327
Description
Expected behavior
The study should be stopped without errors.
Environment
- Optuna version: v2.6.0.dev
- Python version: 3.8.5
- OS: Ubuntu 20.04
- (Optional) Other libraries and their versions: N/A
Error messages, stack traces, or logs
[I 2021-02-08 10:06:01,525] Trial 22 pruned.
Traceback (most recent call last):
File "study-error-2322.py", line 18, in <module>
study.optimize(objective, n_trials=2 ** 4 + 10)
File "/home/yanase/pfn/code/optuna/optuna/study.py", line 378, in optimize
_optimize(
File "/home/yanase/pfn/code/optuna/optuna/_optimize.py", line 66, in _optimize
_optimize_sequential(
File "/home/yanase/pfn/code/optuna/optuna/_optimize.py", line 163, in _optimize_sequential
trial = _run_trial(study, func, catch)
File "/home/yanase/pfn/code/optuna/optuna/_optimize.py", line 237, in _run_trial
study.tell(trial, values=values, state=state)
File "/home/yanase/pfn/code/optuna/optuna/study.py", line 627, in tell
self.sampler.after_trial(study, frozen_trial, state, values)
File "/home/yanase/pfn/code/optuna/optuna/samplers/_grid.py", line 196, in after_trial
study.stop()
File "/home/yanase/pfn/code/optuna/optuna/pruners/_hyperband.py", line 306, in __getattribute__
raise AttributeError(
AttributeError: _BracketStudy does not have attribute of 'stop'Reproducible examples (optional)
import optuna
def objective(trial):
total = 0
for i in range(4):
x = trial.suggest_categorical(f"x{i}", (0, 1))
total += x
trial.report(total, i)
if trial.should_prune():
raise optuna.TrialPruned()
return total
if __name__ == "__main__":
sampler = optuna.samplers.GridSampler(search_space={f"x{i}": (0, 1) for i in range(4)})
study = optuna.create_study(sampler=sampler, pruner=optuna.pruners.HyperbandPruner())
study.optimize(objective, n_trials=2 ** 4 + 10)
print(study.trials_dataframe())Additional context (optional)
Cause
This is mainly because _BracketStudy does not have the stop attribute since it was filtered out by _VALID_ATTRS:
optuna/optuna/pruners/_hyperband.py
Lines 271 to 283 in 533f081
When I simply add stop to _VALID_ATTRS, another error occured as follows:
[I 2021-02-08 10:07:20,648] Trial 15 finished with value: 4.0 and parameters: {'x0': 1, 'x1': 1, 'x2': 1, 'x3': 1}. Best is trial 2 with value: 0.0.
Traceback (most recent call last):
File "study-error-2322.py", line 18, in <module>
study.optimize(objective, n_trials=2 ** 4 + 10)
File "/home/yanase/pfn/code/optuna/optuna/study.py", line 378, in optimize
_optimize(
File "/home/yanase/pfn/code/optuna/optuna/_optimize.py", line 66, in _optimize
_optimize_sequential(
File "/home/yanase/pfn/code/optuna/optuna/_optimize.py", line 163, in _optimize_sequential
trial = _run_trial(study, func, catch)
File "/home/yanase/pfn/code/optuna/optuna/_optimize.py", line 237, in _run_trial
study.tell(trial, values=values, state=state)
File "/home/yanase/pfn/code/optuna/optuna/study.py", line 627, in tell
self.sampler.after_trial(study, frozen_trial, state, values)
File "/home/yanase/pfn/code/optuna/optuna/samplers/_grid.py", line 196, in after_trial
study.stop()
File "/home/yanase/pfn/code/optuna/optuna/study.py", line 785, in stop
if self._optimize_lock.acquire(False):
File "/home/yanase/pfn/code/optuna/optuna/pruners/_hyperband.py", line 307, in __getattribute__
raise AttributeError(
AttributeError: _BracketStudy does not have attribute of '_optimize_lock'Further investigation is required.
CI
The error can be seen in the CI log. This error occurs stochastically.
> study.optimize(objective, n_trials=10)
tests/pruners_tests/test_hyperband.py:213:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
optuna/study.py:387: in optimize
show_progress_bar=show_progress_bar,
optuna/_optimize.py:76: in _optimize
progress_bar=progress_bar,
optuna/_optimize.py:163: in _optimize_sequential
trial = _run_trial(study, func, catch)
optuna/_optimize.py:237: in _run_trial
study.tell(trial, values=values, state=state)
optuna/study.py:627: in tell
self.sampler.after_trial(study, frozen_trial, state, values)
optuna/samplers/_grid.py:196: in after_trial
study.stop()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <optuna.pruners._hyperband.HyperbandPruner._create_bracket_study.<locals>._BracketStudy object at 0x7fa8ced1d6d0>
attr_name = 'stop'
def __getattribute__(self, attr_name): # type: ignore
if attr_name not in _BracketStudy._VALID_ATTRS:
raise AttributeError(
> "_BracketStudy does not have attribute of '{}'".format(attr_name)
)
E AttributeError: _BracketStudy does not have attribute of 'stop'
optuna/pruners/_hyperband.py:307: AttributeError