Use cached trials in `TPESampler`'s `sample_relative` by not522 · Pull Request #6265 · optuna/optuna

not522 · 2025-09-01T23:05:54Z

Motivation

Currently, whether TPESampler uses use_cache in study._get_trials depends solely on whether it uses constant liar. However, since sample_relative is called only once at the beginning of a Trial, it can utilize use_cache.

Description of the changes

Making TPESampler's sample_relative method always use cached trials.

Benchmark

import optuna

def objective(trial: optuna.Trial) -> float:
    x = trial.suggest_float("x", -100, 100)
    y = trial.suggest_int("y", -100, 100)
    return x**2 + y**2

sampler = optuna.samplers.TPESampler(seed=42, multivariate=True, constant_liar=True)
study = optuna.create_study(sampler=sampler, storage="sqlite:///tmp.db")
study.optimize(objective, n_trials=1000)

master	PR
13.389s	12.247s

codecov · 2025-09-01T23:15:03Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.16%. Comparing base (57fade1) to head (b9bb801).
⚠️ Report is 494 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6265      +/-   ##
==========================================
- Coverage   89.24%   89.16%   -0.08%     
==========================================
  Files         208      208              
  Lines       13821    13833      +12     
==========================================
  Hits        12334    12334              
- Misses       1487     1499      +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

nabenabe0928 · 2025-09-02T02:28:29Z

@sawa3030
Could you review this PR?

optuna/samplers/_tpe/sampler.py

nabenabe0928

Thank you for the PR, mostly LGTM!
I left some minor comments:)

Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com>

nabenabe0928

LGTM

sawa3030

LGTM

This reverts commit 6027868, reversing changes made to d369746.

Use cached trials in TPESampler's sample_relative

0e488b8

not522 added the enhancement Change that does not break compatibility and not affect public interfaces, but improves performance. label Sep 1, 2025

nabenabe0928 assigned sawa3030 and nabenabe0928 Sep 2, 2025