Skip to content

Releases: optuna/optuna

v4.8.0

16 Mar 04:59
@c-bata c-bata
v4.8.0
689c62d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.8.0 Latest
Latest

This is the release note of v4.8.0.

Highlights

Support for Constant Liar Strategy to GPSampler

A constant liar strategy for efficient parallelization has been introduced to GPSampler by @sawa3030. The figures (left: v4.7.0, right: v4.8.0) show that the overlap of search points has decreased, and a wider variety of solutions are being explored. The experiment uses n_jobs = 10 and n_trials = 100. Currently, this feature supports single-objective and unconstrained optimization. Further extensions are coming in v4.9.0.

v4.7.0 v4.8.0
image60 image26

SHAP-like beeswarm plot visualization

image52

@yasumorishima introduces the new visualization to OptunaHub. Please refer to https://hub.optuna.org/visualization/plot_beeswarm/ for details.

New Features

Enhancements

  • Validate artifact_id in FileSystemArtifactStore to prevent path traversal (#6432, thanks @RinZ27!)
  • fix: correct inverted warning message in pareto front plot (#6498, thanks @aerosta!)

Bug Fixes

  • Fix shared callback state in parallel OptunaSearchCV with LightGBM (optuna/optuna-integration#260, thanks @Quant-Quasar!)
  • Fix GPSampler crash when torch default device is CUDA (#6418, thanks @VedantMadane!)
  • Fix combination between PartialFixedSampler and TPESampler with group decomposed search space (#6428)
  • Fix TPESampler with multivariate and constant_liar (#6505)

Documentation

  • Add documentation indicating that WilcoxonPruner requires scipy (#6477)
  • Remove version & language selectors in the sidebar of docs (#6482)

Examples

Tests

  • Add SamplerTestCase class in optuna.testing package (#6424)
  • Move test_before_trial and test_after_trial_* to test_trial.py and test_study.py, respectively (#6429)

Code Fixes

  • Move type-only imports to TYPE_CHECKING in _param_importances.py (#6423, thanks @dotz0ver!)
  • Use future.annotations in matplotlib/_intermediate_values (#6425, thanks @Lakshman142!)
  • Reformat files with the latest version of ruff (#6426)
  • Replace .format() with f-strings in _parallel_coordinate.py (#6431, thanks @yasumorishima!)
  • Use TYPE_CHECKING for import in pruners/_base.py (#6434, thanks @yasumorishima!)
  • Use f-strings in optuna/storages/_base.py (#6435, thanks @edwiniac!)
  • Use f-strings in _contour.py (#6436, thanks @edwiniac!)
  • Use f-strings in _intermediate_values.py (#6437, thanks @edwiniac!)
  • Use f-strings in cli.py (#6438, thanks @edwiniac!)
  • Use f-strings in optuna/testing/storages.py (#6439, thanks @edwiniac!)
  • Use f-strings in optuna/storages/journal/_storage.py (#6440, thanks @edwiniac!)
  • Use f-strings in _imports.py (#6442, thanks @edwiniac!)
  • Use f-strings in _deprecated.py (#6443, thanks @edwiniac!)
  • Use TYPE_CHECKING for import in pruners/_hyperband.py (#6447, thanks @yasumorishima!)
  • Use TYPE_CHECKING for typing-only imports in test_timeline.py (#6451, thanks @KRMed!)
  • Refactor: Use f string in multi_objective tutorial (#6455, thanks @ryota717!)
  • refactor: move type-only imports into TYPE_CHECKING in samplers/_grid.py (#6466, thanks @dhruvildarji!)
  • refactor: move BaseDistribution into TYPE_CHECKING in search_space/intersection.py (#6467, thanks @dhruvildarji!)
  • Simplify Union type alias in optuna/samplers/_cmaes.py (#6478, thanks @roli-lpci!)
  • Use f-string in _convert_positional_args.py (#6483, thanks @toroleapinc!)
  • Add acqf import inside a TYPE_CHECKING block (#6484, thanks @acabellom!)
  • Refactor __repr__ in trial/_frozen.py to use f-strings (#6485, thanks @Bhavyag1337!)
  • Use TYPE_CHECKING in pruners/_threshold.py (#6487, thanks @LuciferDono!)
  • Use f-string in tutorial/10_key_features/005_visualization.py (#6489, thanks @maheer14!)
  • Move JSONSerializable import to TYPE_CHECKING in study/study.py (#6490, thanks @yasumorishima!)
  • Use f-string and {var_name=} instead of .format and var_name={var_name} (#6494, thanks @buddy0452004!)
  • Move optuna import to TYPE_CHECKING in pruners/_patient.py (#6501, thanks @yasumorishima!)
  • Move application import to TYPE_CHECKING in pruners/_patient.py (#6502, thanks @nightcityblade!)
  • Move application import to TYPE_CHECKING in pruners/_successive_halving.py (#6503, thanks @nightcityblade!)
  • Replace .format() with f-strings in 002_configurations.py (#6506, thanks @acabellom!)
  • Fix type checking (#6507, thanks @sateeshkumarb!)

Continuous Integration

Other

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Bhavyag1337, @KRMed, @Lakshman142, @LuciferDono, @ParagEkbote, @Quant-Quasar, @RinZ27, @VedantMadane, @acabellom, @aerosta, @buddy0452004, @c-bata, @dhruvildarji, @dotz0ver, @edwiniac, @gen740, @kAIto47802, @maheer14, @nabenabe0928, @nightcityblade, @not522, @roli-lpci, @ryota717, @sateeshkumarb, @sawa3030, @toroleapinc, @y0z, @yasumorishima

Contributors

sateeshkumarb, not522, and 26 other contributors
Assets 2
Loading
0 Join discussion

v4.7.0

19 Jan 05:45
@c-bata c-bata
v4.7.0
40703c5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.7.0

This is the release note of v4.7.0.

Highlights

Two New Multi-Objective Samplers Added to OptunaHub!

hype-sampler

@hrntsm introduces two new multi-objective samplers—SPEA-II (Strength Pareto Evolutionary Algorithm 2) and HypE (Hypervolume Estimation Algorithm)—to OptunaHub. SPEA-II is an improved multi-objective evolutionary algorithm that differs from NSGA-II in its selection mechanism. HypE is a fast, hypervolume-based evolutionary algorithm designed for many-objective optimization problems. Please refer to the following pages for more details:

PedAnovaImportanceEvaluator Now Supports Local Hyperparameter Importance Computation

The target_quantile and region_quantile arguments have been introduced to PedAnovaImportanceEvaluator. This change allows you to investigate local hyperparameter importance rather than the global one with region_quantile < 1.0. See the original paper for the technical details.

Enhancements

  • Introduce stacklevel-aware custom warnings (#6293)
  • Cache distributions to skip consistency check (#6301)
  • Add warnings when JournalStorage lock acquisition is delayed (#6361)
  • Add support for local HPI in PED-ANOVA (#6362)

Bug Fixes

  • Fix log PDF of discrete trunc log-norm distribution for TPESampler (#6258)
  • Fix coefficient in PED-ANOVA (#6358)
  • Fix GPSampler crash when default torch device is CUDA (#6397, thanks @Quant-Quasar!)

Documentation

  • Add SECURITY.md (#6317)
  • Add a note for future dev of exclusive HV (#6318)
  • Update GPSampler documentation to include D-BE optimization details (#6347, thanks @Kaichi-Irie!)
  • Revert PR #6354 to enable -W option on Sphinx build (#6373)

Examples

Tests

  • Fix TC006 violation in tests/visualization_tests/test_utils.py (#6387, thanks @jiayusu!)

Code Fixes

  • Replace .format() with f-string in _setup_studies (#6326, thanks @haitham404!)
  • Update _upload.py for TYPE_CHECKING (#6327, thanks @satyarth7srivastava!)
  • Replace .format() with f-string in progress_bar.py (#6328, thanks @Nayil97!)
  • Use f-strings in optuna/samplers/_cmaes.py (#6331, thanks @swativdusane!)
  • Replace .format() with f-string in _parallel_coordinate.py (#6333, thanks @satyarth7srivastava!)
  • Refactor/fstring storage rdb (#6336, thanks @gadmin7!)
  • Migrate to ruff from black/blackdoc/isort/flake8 (#6341)
  • Replace .format with f-strings in optuna/importance/_base (#6342, thanks @VihaanMotwani!)
  • updated _terminator_improvement.py for TYPE_CHECKING (#6343, thanks @satyarth7srivastava!)
  • Replace .format with f-string in _param_importances.py (#6345, thanks @Harshadev-24!)
  • Replace .format() with f-strings in several modules (#6348, thanks @varundevr!)
  • Replace more .format() calls with f-strings (#6349, thanks @varundevr!)
  • Replace .format with f-string in tutorial/20_recipes/004_cli.py (#6350, thanks @RektPunk!)
  • Replace .format with f-string in optuna/study/_optimize.py (#6351, thanks @RektPunk!)
  • Format optuna/ files with Ruff (#6352)
  • Refactor: Use f-string in 001_rdb.py (#6356, thanks @sotagg!)
  • Format tests/ and tutorials/ files with Ruff (#6360)
  • Remove redundant _color_supported() check (#6363)
  • Add StorageTestCase class in optuna.testing package (#6369)
  • Change string formatting in optuna/pruners/_hyperband.py (#6370, thanks @eleannapapaio!)
  • Refactor test_study.py to use f-string instead of .format() (#6372, thanks @nepersoned!)
  • Fix mypy error for np.select (#6374)
  • Change string formatting for _successive_halving.py (#6375, thanks @spenam!)
  • Fix type annotations for optuna/trial/_frozen.py (#6377, thanks @spenam!)
  • fix type annotations for optuna/study/study.py (#6378, thanks @spenam!)
  • Fix type annotations for tests/study_tests/test_study.py (#6379, thanks @spenam!)
  • Move type-only imports to TYPE_CHECKING in test_visualizations.py (#6380, thanks @Sip4818!)
  • Move type-only imports to TYPE_CHECKING in _constrained_optimization.py (#6381, thanks @Sip4818!)
  • Replace format with f-string (#6383, thanks @varundevr!)
  • Move type-only imports to TYPE_CHECKING in _multi_objective.py (#6385, thanks @Sip4818!)
  • Move FrozenTrial import under TYPE_CHECKING for _study_summary.py file (#6386, thanks @Sip4818!)
  • Move type-check imports to TYPE_CHECKING in optuna/terminator/callback.py (#6388, thanks @Sip4818!)
  • Using f-string instead of .format() (#6389, thanks @Lakshman142!)
  • Use f-strings in optuna/_experimental.py (#6390, thanks @Rohan0497!)
  • Fix invalid StorageTestCase scenarios involving trial state and values (#6391)
  • Move type-hint import inside Type-Checking block in optuna\terminator\erroreval.py (#6395, thanks @Sip4818!)
  • Move type-check imports to TYPE_CHECKING in optuna\terminator\improvement\emmr.py (#6396, thanks @Sip4818!)
  • Move typing-only imports under TYPE_CHECKING in matplotlib/_slice.py (#6399, thanks @kapishyadav!)
  • Use logger.warning instead of optuna_warn for lock-acquisition delay notifications (#6400)
  • Update string formatting in optuna/samplers/_grid.py (#6401, thanks @kapishyadav!)
  • Updating storages/_in_memory.py to use f-strings (#6404, thanks @jrings!)
  • Move type-hint imports into type-checking block in optuna\terminator\improvement\evaluator.py (#6405, thanks @Sip4818!)
  • Move type-hint imports into type-checking block in median_erroreval.py (#6408, thanks @Sip4818!)
  • Replace .format() with f-string in _rank.py (#6409, thanks @jwalith!)
  • Replace .format() with f-string in test_hyperband.py (#6411, thanks @Banjiola!)
  • Replace .format() with f-string in _fixed.py (#6412, thanks @VedantMadane!)

Continuous Integration

Other

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @Banjiola, @Harshadev-24, @HideakiImamura, @Kaichi-Irie, @Lakshman142, @Nayil97, @ParagEkbote, @Quant-Quasar, @RektPunk, @Rohan0497, @Sip4818, @VedantMadane, @VihaanMotwani, @c-bata, @eleannapapaio, @fritshermans, @fusawa-yugo, @gadmin7, @gen740, @haitham404, @jiayusu, @jrings, @jwalith, @kAIto47802, @kapishyadav, @nabenabe0928, @nepersoned, @not522, @nzw0301, @satyarth7srivastava, @sawa3030, @sotagg, @spenam, @swativdusane, @toshihikoyanase, @varundevr, @y0z

Contributors

not522, jrings, and 37 other contributors
Loading
0 Join discussion

v4.6.0

10 Nov 05:13
@y0z y0z
v4.6.0
3da7487
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.6.0

This is the release note of v4.6.0.

Highlights

Optuna Dashboard LLM Integration

Optuna Dashboard is a web-based tool that helps you easily explore and visualize your Optuna optimization history. The latest release, v0.20.0, introduces LLM integration, enabling the natural language-based Trial filtering and automatic Plotly chart generation. Please refer to the release blog for more details.

image55

Further Speed Enhancements for GPSampler

GPSampler becomes significantly faster owing to parallelized multi-start acquisition function optimization via PyTorch batching, and to optimized NumPy operations.

image15

Full Support for Multi-objective and Constrained Optimization in AutoSampler

We have fully implemented sampler selection rules for multi-objective and constrained optimization in AutoSampler. For more details, please see our blog post, "AutoSampler: Full Support for Multi-Objective & Constrained Optimization."

optuna-blog-autosampler-multi-constrained

Additions of Robust Bayesian Optimization Packages

Robust Bayesian optimization methods have been added to OptunaHub. Robust Bayesian optimization enables suggesting more robust parameters against input perturbations. This is especially helpful for Sim2Real transfer scenarios.

image25

Breaking Changes

Enhancements

  • Use iterator for lazy evaluation in journal storage’s read_logs (#6144)
  • Cache pair-wise distances to speed up GPSampler (#6244)
  • Speed up LogEI implementation (#6248)
  • Speed up EHVI by optimizing tensor operation order (#6257)
  • Use the decremental approach in the hypervolume contribution calculation (#6264)
  • Use cached trials in TPESampler's sample_relative (#6265)
  • Remove find_or_raise_by_id in _set_trial_value_without_commit (#6266)
  • Speed up GPSampler by Batching Acquisition Function Evaluations (#6268, thanks @Kaichi-Irie!)
  • Use cached study direction and trial for _CachedStorage's get_best_trial (#6270)
  • Add upsert in _set_trial_attr_without_commit for PostgreSQL (#6282, thanks @jaikumarm!)
  • Add states argument to _read_trials_from_remote_storage (#6288)
  • Use cached trials for intersection search space calculation (#6291)
  • Replace np.linalg.inv with np.linalg.cholesky to speed up GPSampler for numpy>=2.0.0 (#6296)

Bug Fixes

  • Skip trial validation on copy_study (#6249)
  • Fix incremental update algorithm in _CachedStorage's _read_trials_from_remote_storage (#6310)
  • Add safety guard for exhaustive search (#6321)

Documentation

  • Add AutoSampler to the sampler comparison table in the API reference (#6260, thanks @Kaichi-Irie!)
  • Update the GPSampler document to reflect support for constrained multi-objective optimization (#6262)
  • Add a link to the metric TPE paper in the TPESampler document (#6263)
  • Update announcement (#6285)
  • Update the table of Samplers in docs (#6287, thanks @fusawa-yugo!)
  • Fix the table of samplers in the docs (#6290)

Examples

Tests

Code Fixes

  • Update target version of black from Python 3.8 to 3.9 (optuna/optuna-integration#254)
  • Move fit_kernel_params to GPRegressor (#6243)
  • Modify TYPE_CHECKING in _brute_force.py (#6259, thanks @Kaichi-Irie!)
  • Move SciPy to the lazy import section in _gp/scipy_blas_thread_patch.py (#6269, thanks @Kaichi-Irie!)
  • Make the interface of batched_lbfgsb module compatible with scipy.optimize (#6273, thanks @Kaichi-Irie!)
  • Fix type checking in optuna.study._frozen.py (#6275, thanks @GabrielRomaoG!)
  • Move typing-only imports under TYPE_CHECKING in optuna.importance.__init__ (#6278, thanks @euangoodbrand!)
  • Move typing-only imports under TYPE_CHECKING in FanovaImportanceEvaluator (#6279, thanks @euangoodbrand!)
  • Move typing-only imports under TYPE_CHECKING in /study/_optimize.py (#6280, thanks @euangoodbrand!)
  • Use TYPE_CHECKING in optuna/pruners/_nop.py (#6297, thanks @AddyM!)
  • Use TYPE_CHECKING in optuna/samplers/_random.py (#6298, thanks @AddyM!)
  • Speed up squared distance computation (#6300)
  • Refactor emmr (#6304)
  • Fix string format of optuna/distributions.py (#6306)
  • Fix string format of tests/samplers_tests/tpe_tests/test_truncnorm.py (#6307)
  • Update black target to Python 3.9 (#6308)
  • Fix string format for optuna/study/study.py (#6309, thanks @unKnownNG!)
  • Changed the old .format code to the new f string format in the test_journal.py (#6312, thanks @Zrahay!)
  • Update string formatting in visualization/_pareto_front.py (#6314, thanks @dross20!)
  • Use f-string in 001_first.py (#6315, thanks @satyarth7srivastava!)
  • Use f-strings in _intermediate_values.py (#6316, thanks @nihalsiddiqui7!)
  • Refactor .format to f-string in _percentile.py (#6323, thanks @Jongwan93!)

Continuous Integration

Other

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@AddyM, @GabrielRomaoG, @Jongwan93, @Kaichi-Irie, @ParagEkbote, @Zrahay, @c-bata, @contramundum53, @dross20, @euangoodbrand, @fusawa-yugo, @gen740, @jaikumarm, @kAIto47802, @ktns, @nabenabe0928, @nihalsiddiqui7, @not522, @satyarth7srivastava, @sawa3030, @toshihikoyanase, @unKnownNG, @y0z

Contributors

jaikumarm, not522, and 21 other contributors
Loading
0 Join discussion

v4.5.0

18 Aug 06:48
@y0z y0z
v4.5.0
d7e1c1b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.5.0

This is the release note of v4.5.0.

Highlights

GPSampler for constrained multi-objective optimization

GPSampler is now able to handle multiple objective and constraints simultaneously using the newly introduced constrained LogEHVI acquisition function.

The figures below show the difference between GPSampler (LogEHVI, unconstrained) vs GPSampler (constrained LogEHVI, new feature). The 3-dimensional version of the C2DTLZ2 benchmark problem we used is a problem where some areas of the Pareto front of the original DTLZ2 problem are made infeasible by constraints. Therefore, even if constraints are not taken into account, it is possible to obtain the Pareto front. Experimental results show that both LogEHVI and constrained LogEHVI can approximate the Pareto front, but the latter has significantly fewer infeasible solutions, demonstrating its efficiency.

Optuna v4.4 (LogEHVI) Optuna v4.5 (Constrained LogEHVI)
Log EHVI Constrained LogEHVI

Significant speedup of TPESampler

TPESampler is significantly (about 5x as listed in the table below) faster! It enables a larger number of trials in each study. The speedup was achieved through a series of enhancements in constant factors.

The following table shows the speed comparison of TPESampler between v4.4.0 and v4.5.0. The experiments were conducted using multivariate=True on a search space with 3 continuous parameters and 3 numerical discrete parameters. Each row shows the runtime for each number of objectives and each column shows each number of trials to be evaluated. Each runtime is shown along with the standard error over 3 random seeds. The numbers in parentheses represent the speedup factor in comparison to v4.4.0. For example, (5.1x) means the runtime of v4.5.0 is 5.1 times faster than that of v4.4.0.

n_objectives/n_trials 500 1000 1500 2000
1 1.4 $\pm$ 0.03 (5.1x) 3.9 $\pm$ 0.07 (5.3x) 7.3 $\pm$ 0.09 (5.4x) 11.9 $\pm$ 0.10 (5.4x)
2 1.8 $\pm$ 0.01 (4.7x) 4.7 $\pm$ 0.02 (4.8x) 8.7 $\pm$ 0.03 (4.8x) 13.9 $\pm$ 0.04 (4.9x)
3 2.0 $\pm$ 0.01 (4.2x) 5.4 $\pm$ 0.03 (4.4x) 10.0 $\pm$ 0.03 (4.6x) 15.9 $\pm$ 0.03 (4.7x)
4 4.2 $\pm$ 0.11 (3.2x) 12.1 $\pm$ 0.14 (3.9x) 20.9 $\pm$ 0.23 (4.2x) 31.3 $\pm$ 0.05 (4.4x)
5 12.1 $\pm$ 0.59 (4.7x) 30.8 $\pm$ 0.16 (5.8x) 50.7 $\pm$ 0.46 (6.5x) 72.8 $\pm$ 1.13 (7.1x)

Significant speedup of plot_hypervolume_history

plot_hypervolume_history is essential to assess the performance of multi-objective optimization, but it was unbearably slow when a large number of trials are evaluated on a many-objective (The number of objectives > 3) problem. v4.5.0 addressed this issue by incrementally updating the hypervolume instead of calculating each hypervolume from scratch.

The following figure shows the elapsed times of hypervolume history plot in Optuna v4.4.0 and v4.5.0 using a four-objective problem. The x-axis represents the number of trials and the y-axis represents the elapsed times for each setup. The blue and red lines are the results of v4.4.0 and v4.5.0, respectively.

Speedup of plot_hypervolume_history

CmaEsSampler now supports 1D search space

Up until Optuna v4.4, CmaEsSampler could not handle one-dimensional space and fell back to random search. Optuna v4.5 now allows the CMA-ES algorithm to be used for one-dimensional space.

The optunahub library is available on conda-forge

Now, you can install the optunahub library via conda-forge as follows.

conda install conda-forge::optunahub
Conda-Forge

New Features

  • Add ConstrainedLogEHVI (#6198)
  • Add support for constrained multi-objective optimization in GPSampler (#6224)
  • Support 1D Search Spaces in CmaEsSampler (#6228)

Enhancements

  • Move optuna._lightgbm_tuner module (optuna/optuna-integration#233, thanks @milkcoffeen!)
  • Fix numerical issue warning on qehvi_candidates_func (optuna/optuna-integration#242, thanks @LukeGT!)
  • Calculate hypervolume in HSSP using sum of contributions (#6130)
  • Use hypervolume difference as upperbound of contribs in HSSP (#6131)
  • Refactor tell_with_warning to avoid unnecessary get_trial call (#6133)
  • Print fully qualified name of experimental function by default (#6162, thanks @ktns!)
  • Include scipy-stubs in the type-check dependencies (#6174, thanks @jorenham!)
  • Warn when GPSampler falls back to RandomSampler (#6179, thanks @sisird864!)
  • Handle slowdown of GPSampler due to L-BFGS in SciPy v1.15 (#6191)
  • Use the Newton method instead of bisect in ndtri_exp (#6194)
  • Speed up erf for TPESampler (#6200)
  • Avoid duplications in _log_gauss_mass evaluations (#6202)
  • Remove unnecessary NumPy usage (#6215)
  • Use subset comparator to judge if trials are included in search space (#6218)
  • Speed up log pdf in _BatchedTruncNormDistributions by vectorization (#6220)
  • Speed up WFG by skipping is_pareto_front and using simple Python loops (#6223)
  • Vectorize ndtri_exp (#6229)
  • Speed up plot_hypervolume_history (#6232)
  • Speed up HSSP 4D+ by using a decremental approach (#6234)
  • Use lru_cache to skip HSSP (#6240, thanks @fusawa-yugo!)
  • Add hypervolume computation for a zero size array (#6245)

Bug Fixes

  • Fix: Resolve PG17 incompatibility for ENUMS in CASE statements (#6099, thanks @vcovo!)
  • Fix ill-combination of journal and gRPC (#6175)
  • Fix a bug in constrained GPSampler (#6181)
  • Fix TPESampler with multivariate and constant_liar (#6189)

Installation

  • Remove version constraint for torch with Python 3.13 (#6233)

Documentation

  • Add missing spaces in error message about inconsistent intermediate values (optuna/optuna-integration#239, thanks @Greesb!)
  • Improve parallelization document (#6123)
  • Add FAQ for case sensitivity problem with MySQL (#6127, thanks @fusawa-yugo!)
  • Add an FAQ entry about specifying optimization parameters (#6157)
  • Update link to survey (#6169)
  • Add introduction of OptunaHub in docs (#6171, thanks @fusawa-yugo!)
  • Add GPSampler as a sampler that supports constraints (#6176, thanks @1kastner!)
  • Remove optuna-fast-fanova references from documentation (#6178)
  • Fix stale docs in GP-related modules (#6184)
  • Embed link to OptunaHub in documentation (#6192, thanks @fusawa-yugo!)
  • Update README.md (#6222, thanks @muhammadibrahim313!)
  • Add a note about unrelated changes in PR (#6226)
  • Document the default evaluator in Optuna Dashboard (#6238)

Examples

Tests

  • Fix test_log_completed_trial_skip_storage_access (#6208)

Code Fixes

  • Clean up GP-related docs (#6125)
  • Refactor return style #6136 (#6151, thanks @unKnownNG!)
  • Refactor KernelParamsTensor towards cleaner GP-related modules (#6152)
  • Rename KernelParamsTensor to GPRegressor (#6153)
  • Refactor returns in v3.0.0.d.py (#6154, thanks @dross20!)
  • Refactor acquisition function minimally (#6166)
  • Implement Type-Checking for optuna/_imports.py (#6167, thanks @AdrianStrymer!)
  • Fix type checking in optuna.artifacts._download.py (#6177, thanks @dross20!)
  • Integrate is_categorical to search space (#6182)
  • Fix type checking in optuna.artifacts._list_artifact_meta.py (#6187, thanks @dross20!)
  • Introduce the independent sampling warning template (#6188)
  • Use warning template for independent sampling in GPSampler (#6195)
  • Refactor SearchSpace in GP (#6197)
  • Refactor _truncnorm (#6201)
  • Use TYPE_CHECKING in optuna/_gp/acqf.py to avoid circular imports (#6204, thanks @CarvedCoder!)
  • Use TYPE_CHECKING in optuna/_gp/optim_mixed.py to avoid circular imports (#6205, thanks @Subodh-12!)
  • Flip the sign of constraints in GPSampler (#6213)
  • Implement NSGA-III using BaseGASampler (#6219)
  • Replace torch.newaxis with None for old PyTorch (#6237)

Continuous Integration

Other

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@1kastner, @AdrianStrymer, @CarvedCoder, @Greesb, @HideakiImamura, @LukeGT, @ParagEkbote, @Subodh-12, @c-bata, @contramundum53, @dhyeyinf, @dross20, @fusaw...

Read more

Contributors

not522, ktns, and 27 other contributors
Loading
0 Join discussion

v4.4.0

16 Jun 05:12
@HideakiImamura HideakiImamura
v4.4.0
0742587
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.4.0

This is the release note of v4.4.0.

Highlights

In addition to new features, bug fixes, and improvements in documentation and testing, version 4.4 introduces a new tool called the Optuna MCP Server.

Optuna MCP Server

The Optuna MCP server can be accessed by any MCP client via uv — for instance, with Claude Desktop, simply add the following configuration to your MCP server settings file. Of course, other LLM clients like VSCode or Cline can also be used similarly. You can also access it via Docker. If you want to persist the results, you can use the — storage option. For details, please refer to the repository.

{
  "mcpServers": {
    … (Other MCP Servers' settings)
    "Optuna": {
      "command": "uvx",
      "args": [
        "optuna-mcp"
      ]
    }
  }
}

image3

Gaussian Process-Based Multi-objective Optimization

Optuna’s GPSampler, introduced in version 3.6, offers superior speed and performance compared to existing Bayesian optimization frameworks, particularly when handling objective functions with discrete variables. In Optuna v4.4, we have extended this GPSampler to support multi-objective optimization problems. The applications of multi-objective optimization are broad, and the new multi-objective capabilities introduced in this GPSampler are expected to find applications in fields such as material design, experimental design problems, and high-cost hyperparameter optimization.

GPSampler can be easily integrated into your program and performs well against the existing BoTorchSampler. We encourage you to try it out with your multi-objective optimization problems.

sampler = optuna.samplers.GPSampler()
study = optuna.create_study(directions=["minimize", "minimize"], sampler=sampler)

image2

New Features in OptunaHub

During the development period of Optuna v4.4, several new features were also introduced to OptunaHub, the feature-sharing platform for Optuna:

Vizier sampler performance
image1
TPE acquisition visualizer
image4

Breaking Changes

  • Update consider_prior Behavior and Remove Support for False (#6007)
  • Remove restart_strategy and inc_popsize to simplify CmaEsSampler (#6025)
  • Make all arguments of TPESampler keyword-only (#6041)

New Features

  • Add a module to preprocess solutions for hypervolume improvement calculation (#6039)
  • Add AcquisitionFuncParams for LogEHVI (#6052)
  • Support Multi-Objective Optimization GPSampler (#6069)
  • Add n_recent_trials to plot_timeline (#6110, thanks @msdsm!)

Enhancements

  • Adapt TYPE_CHECKING of samplers/_gp/sampler.py (#6059)
  • Avoid deepcopy in _tell_with_warning (#6079)
  • Add _compute_3d for hypervolume computation (#6112, thanks @shmurai!)
  • Improve performance of plot_hypervolume_history (#6115, thanks @shmurai!)
  • add deprecated/removed version specification to calls of convert_positional_args (#6117, thanks @shmurai!)
  • Optimize Study.best_trial performance by avoiding unnecessary deep copy (#6119, thanks @msdsm!)
  • Refactor and speed up HV3D (#6124)
  • Add assume_pareto for hv calculation in _calculate_weights_below_for_multi_objective (#6129)

Bug Fixes

  • Update vsbx (#6033, thanks @hrntsm!)
  • Fix request.values in OptunaStorageProxyService (#6044, thanks @hitsgub!)
  • Fix a bug in distributed optimization using NSGA-II/III (#6066, thanks @leevers!)
  • Fix: fetch all trials in BruteForceSampler for HyperbandPruner (#6107)

Documentation

Examples

Tests

  • Add float precision tests for storages (#6040)
  • Refactor test_base_gasampler.py (#6104)
  • chore: run tests for importance only with in-memory (#6109)
  • Improve test cases for n_recent_trials of plot_timeline (follow-up #6110) (#6116)
  • Performance optimization for test_study.py by removing redundancy (#6120)

Code Fixes

  • Optional mypy check (#6028)
  • Update Type-Checking for optuna/_experimental.py (#6045, thanks @ParagEkbote!)
  • Update Type-Checking for optuna/importance/_base.py (#6046, thanks @ParagEkbote!)
  • Update Type-Checking for optuna/_convert_positional_args.py (#6050, thanks @ParagEkbote!)
  • Update Type-Checking for optuna/_deprecated.py (#6051, thanks @ParagEkbote!)
  • Update Type-Checking for optuna/_gp/gp.py (#6053, thanks @ParagEkbote!)
  • Add validate eta in sbx (#6056, thanks @hrntsm!)
  • Remove CmaEsAttrKeys and _attr_keys for Simplification (#6068)
  • Replace np.isnan with math.isnan (#6080)
  • Refactor warning handling of _tell_with_warning (#6082)
  • Implement Type-Checking for optuna/distributions.py (#6086, thanks @AdrianStrymer!)
  • Update TYPE_CHECKING for optuna/_gp/gp.py (#6090, thanks @Samarthi!)
  • Support mypy 1.16.0 (#6102)
  • Emit ExperimentalWarning if heartbeat is enabled (#6106, thanks @lan496!)
  • Simplify tuple return in optuna/visualization/_terminator_improvement.py (#6139, thanks @Prashantdhaka23!)
  • Refactor return standardization in optim_mixed.py (#6140, thanks @Ajay-Satish-01!)
  • Simplify tuple return in test_trial.py (#6141, thanks @saishreyakumar!)
  • Refactor return statement style in optuna/storages/_rdb/models.py for consistency among the codebase (#6143, thanks @Shubham05122002!)

Continuous Integration

Other

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@AdrianStrymer, @Ajay-Satish-01, @Alnusjaponica, @Copilot, @HideakiImamura, @ParagEkbote, @Prashantdhaka23, @Samarthi, @Shubham05122002, @SubhadityaMukherjee, @c-bata, @contramundum53, @copilot-pull-request-reviewer[bot], @fusawa-yugo, @gen740, @himkt, @hitsgub, @hrntsm, @kAIto47802, @lan496, @leevers, @milkcoffeen, @msdsm, @nabenabe0928, @not522, @nzw0301, @s...

Read more

Contributors

not522, toshihikoyanase, and 28 other contributors
Loading
0 Join discussion

v4.3.0

14 Apr 05:06
@y0z y0z
v4.3.0
bc23723
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.3.0

This is the release note of v4.3.0.

Highlights

This has various bug fixes and improvements to the documentation and more.

Breaking Changes

Enhancements

  • Accept custom objective in LightGBMTuner (optuna/optuna-integration#203, thanks @sawa3030!)
  • Improve time complexity of IntersectionSearchSpace (#5982, thanks @GittyHarsha!)
  • Add _prev_waiting_trial_number in InMemoryStorage to improve the efficiency of _pop_waiting_trial_id (#5993, thanks @sawa3030!)
  • Add arguments of versions to convert_positional_args (#6009, thanks @fusawa-yugo!)
  • Add wait_server_ready method in GrpcStorageProxy (#6010, thanks @hitsgub!)
  • Remove warning messages for Matplotlib-based plot_contour and plot_rank (#6011)
  • Fix type checking in optuna._callbacks.py (#6030)
  • Enhance SBXCrossover (#6008, thanks @hrntsm!)

Bug Fixes

  • Convert storage into InMemoryStorage before copying to the local (optuna/optuna-integration#213)
  • Fix contour plot of matplotlib (#5892, thanks @fusawa-yugo!)
  • Fix threading lock logic (#5922)
  • Use _LazyImport for grpcio package (#5954)
  • Prevent Lock Blocking by Adding Timeout to JournalStorage (#5971, thanks @sawa3030!)
  • Fix a minor bug in GPSampler for objective that returns inf (#5995)
  • Fix a bug that a gRPC server doesn't work with JournalStorage (#6004, thanks @fusawa-yugo!)
  • Fix _pop_waiting_trial_id for finished trial (#6012)
  • Resolve the issue where BruteForceSampler fails to suggest all combinations (#5893)

Documentation

Examples

Tests

Code Fixes

  • Add BaseGASampler (#5864)
  • Fix comments in pyproject.toml (#5972)
  • Remove FirstTrialOnlyRandomSampler (#5973, thanks @mehakmander11!)
  • Remove _check_and_set_param_distribution (#5975, thanks @siddydutta!)
  • Remove testing/distributions.py (#5977, thanks @mehakmander11!)
  • Remove _StudyInfo's param_distribution in _cached_storage.py (#5978, thanks @tarunprabhu11!)
  • Introduce UpdateFinishedTrialError to raise an error when attempting to modify a finished trial (#6001, thanks @sawa3030!)
  • Deprecate consider_prior in TPESampler (#6005, thanks @sawa3030!)
  • Improve Code Readability by Following PEP8 Standards (#6006, thanks @sawa3030!)
  • Made error message for create_study's direction easier to understand optuna.study (#6021, thanks @sinano1107!)

Continuous Integration

Other

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.

@Alnusjaponica, @GittyHarsha, @HideakiImamura, @ParagEkbote, @c-bata, @contramundum53, @ffineis, @fusawa-yugo, @gen740, @hitsgub, @hrntsm, @kAIto47802, @ktns, @mehakmander11, @nabenabe0928, @not522, @nzw0301, @porink0424, @sawa3030, @siddydutta, @sinano1107, @tarunprabhu11, @toshihikoyanase, @y0z

Contributors

not522, ktns, and 22 other contributors
Loading
0 Join discussion

v4.2.1

12 Feb 07:56
@c-bata c-bata
v4.2.1
2e595ac
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v4.2.1

This is the release note of v4.2.1. This release includes a bug fix addressing an issue where Optuna was unable to import if an older version of the grpcio package was installed.

Bug

  • [backport] Use _LazyImport for grpcio package (#5965)

Other

  • Bump up version number to v4.2.1 (#5964)

Thanks to All the Contributors!

This release was made possible by the authors and the people who participated in the reviews and discussions.
@c-bata @HideakiImamura @nabenabe0928

Contributors

c-bata, HideakiImamura, and nabenabe0928
Loading
0 Join discussion

v3.6.2

27 Jan 07:13
@c-bata c-bata
v3.6.2
c9b0829
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.6.2

This is the release note of v3.6.2.

Bug Fixes

  • [Backport] Fix the default sampler of load_study function.

Other

  • Bump up version number to v3.6.2
Loading

v3.5.1

27 Jan 07:13
@c-bata c-bata
v3.5.1
46a02d6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.5.1

This is the release note of v3.5.1.

Bug Fixes

  • [Backport] Fix the default sampler of load_study function.

Other

  • Bump up version number to v3.5.1.
Loading

v3.4.1

27 Jan 06:51
@c-bata c-bata
v3.4.1
299e162
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.4.1

This is the release note of v3.4.1.

Bug Fixes

  • [Backport] Fix the default sampler of load_study function.

Other

  • Bump up version number to v3.4.1.
Loading