Skip to content

Speed up plot_hypervolume_history#6232

Merged
y0z merged 2 commits intooptuna:masterfrom
not522:speed-up-hypervolume-history
Aug 5, 2025
Merged

Speed up plot_hypervolume_history#6232
y0z merged 2 commits intooptuna:masterfrom
not522:speed-up-hypervolume-history

Conversation

@not522
Copy link
Copy Markdown
Member

@not522 not522 commented Aug 4, 2025

Motivation

plot_hypervolume_history can be accelerated by using incremental updates.

Description of the changes

Changed hypervolume calculation to use incremental updates (S(A v B) = S(A) + S(B) - S(A ^ B)).

Benchmark

master: 213.47 sec
PR: 1.52 sec

import time
import optuna

def objective(trial):
    x = trial.suggest_float("x", 0, 1)
    y = trial.suggest_float("y", 0, 1)
    z = trial.suggest_float("z", 0, 1)
    w = trial.suggest_float("w", 0, 1)
    trial.set_user_attr("constraints", [x ** 2 + y ** 2 + z ** 2 + w ** 2 - 1])
    return x, y, z, w

def constraints_func(trial):
    return trial.user_attrs["constraints"]

sampler = optuna.samplers.NSGAIISampler(seed=42, constraints_func=constraints_func)
study = optuna.create_study(directions=["maximize", "maximize", "maximize", "maximize"], sampler=sampler)
study.optimize(objective, n_trials=10000)

reference_point=[0, 0, 0, 0]
start = time.time()
optuna.visualization.plot_hypervolume_history(study, reference_point)
print(time.time() - start)

@not522 not522 added the enhancement Change that does not break compatibility and not affect public interfaces, but improves performance. label Aug 4, 2025
@nabenabe0928
Copy link
Copy Markdown
Contributor

@sawa3030 Could you review this PR?

Copy link
Copy Markdown
Contributor

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that the PR works as intended with the following code:

Verification Code
from __future__ import annotations

from collections.abc import Sequence
from typing import NamedTuple

import numpy as np
from tqdm import tqdm

import optuna
from optuna._hypervolume import compute_hypervolume
from optuna.study import Study
from optuna.study._study_direction import StudyDirection
from optuna.trial import TrialState
from optuna.visualization._plotly_imports import _imports


def this_pr(
    values_array: np.ndarray, ref_point: np.ndarray,
) -> np.ndarray:
    n_trials = values_array.shape[0]
    hypervolume_values = np.empty(n_trials, dtype=float)
    best_trials_values_normalized: np.ndarray | None = None
    hypervolume = 0.0
    for i in tqdm(range(n_trials)):
        values_normalized = values_array[i, np.newaxis, :]
        if best_trials_values_normalized is not None:
            if (best_trials_values_normalized <= values_normalized).all(axis=1).any(axis=0):
                hypervolume_values[i] = hypervolume
                continue

        hypervolume += np.prod(ref_point - values_normalized)
        if best_trials_values_normalized is None:
            best_trials_values_normalized = values_normalized
        else:
            limited_sols = np.maximum(best_trials_values_normalized, values_normalized)
            hypervolume -= compute_hypervolume(limited_sols, ref_point)
            is_kept = (best_trials_values_normalized < values_normalized).any(axis=1)
            best_trials_values_normalized = np.concatenate(
                [best_trials_values_normalized[is_kept, :], values_normalized], axis=0
            )
        hypervolume_values[i] = hypervolume

    return hypervolume_values


def master(
    values_array: np.ndarray, ref_point: np.ndarray,
) -> np.ndarray:
    n_trials = values_array.shape[0]
    hypervolume_values = np.empty(n_trials, dtype=float)
    best_trials_values_normalized: np.ndarray | None = None
    for i in tqdm(range(n_trials)):
        values_normalized = values_array[i, np.newaxis, :]
        if best_trials_values_normalized is not None:
            if (best_trials_values_normalized <= values_normalized).all(axis=1).any(axis=0):
                hypervolume_values[i] = hypervolume_values[i - 1]
                continue

        if best_trials_values_normalized is None:
            best_trials_values_normalized = values_normalized
        else:
            is_kept = (best_trials_values_normalized < values_normalized).any(axis=1)
            best_trials_values_normalized = np.concatenate(
                [best_trials_values_normalized[is_kept, :], values_normalized], axis=0
            )
        hypervolume_values[i] = compute_hypervolume(best_trials_values_normalized, ref_point)

    return hypervolume_values


if __name__ == "__main__":
    rng = np.random.RandomState(42)
    values_array = rng.normal(size=(5000, 4))
    ref_point = np.full(4, 10.0)
    out = this_pr(values_array.copy(), ref_point)
    ans = master(values_array.copy(), ref_point)
    assert np.allclose(out, ans)

Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com>
@nabenabe0928
Copy link
Copy Markdown
Contributor

Let me re-assign from @sawa3030 to @y0z !

Copy link
Copy Markdown
Contributor

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@nabenabe0928 nabenabe0928 added this to the v4.5.0 milestone Aug 5, 2025
@nabenabe0928 nabenabe0928 removed their assignment Aug 5, 2025
Copy link
Copy Markdown
Member

@y0z y0z left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@not522
The change works as expected. LGTM.

NIT:
Since values appear to be unnormalized and not guaranteed to be in the [0, 1] range in the general case, I thought the names values_normalized and best_trials_values_normalized might be inappropriate. However, these names do not affect the actual logic, so a fix could be considered in a follow-up PR.

@y0z y0z merged commit cfd3ec3 into optuna:master Aug 5, 2025
15 checks passed
@y0z y0z removed their assignment Aug 5, 2025
@not522 not522 deleted the speed-up-hypervolume-history branch August 5, 2025 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Change that does not break compatibility and not affect public interfaces, but improves performance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants