Motivation

The default sampler, TPESampler, has options expected to improve optimization performance in many situations. For Optuna v3, we consider changing the default options to enjoy effective algorithms.

Description

• Collect comprehensive benchmarks for the important options, e.g., multivariate, group, and constant_liar
• Identify for the cause of the reported benchmark results TPE performs poorly compared to the TPE in BOHB #2871
• Conduct speed benchmarks to confirm the speed is acceptable
• Add the specification of the behavior of n_startup_trials for constant_liar option #3908
• Stabilize multivariate, group, and constant_liar
• Change the behavior to keep the relative sampling even if the range of suggestion changes