Skip to content

Speed up squared distance computation#6300

Merged
sawa3030 merged 4 commits intooptuna:masterfrom
nabenabe0928:speed-up-sqdist
Oct 17, 2025
Merged

Speed up squared distance computation#6300
sawa3030 merged 4 commits intooptuna:masterfrom
nabenabe0928:speed-up-sqdist

Conversation

@nabenabe0928
Copy link
Copy Markdown
Contributor

@nabenabe0928 nabenabe0928 commented Oct 14, 2025

Motivation

This PR speeds up GPSampler by 20% for 300 Trials.

Description of the changes

  • Use matmul instead of prod-sum.

This PR passes the following test:

from __future__ import annotations

import torch


torch.manual_seed(0)


def verify(x1_shape: tuple[int, ...], x2_shape: tuple[int, ...]) -> None:
    assert x1_shape[-1] == x2_shape[-1]
    x1 = torch.randn(x1_shape, dtype=torch.float64)
    x2 = torch.randn(x2_shape, dtype=torch.float64)
    w = torch.rand(x1_shape[-1], dtype=torch.float64)
    sqd = (x1.unsqueeze(-2) - x2.unsqueeze(-3)).square_()
    ans = (w * sqd).sum(dim=-1)
    out = sqd.matmul(w)
    assert torch.allclose(ans, out)
    print(f"{x1_shape=}, {x2_shape}: Successfull")


for i in range(30):
    print(f"Seed{i}")
    for x1_shape in [(4,), (3, 4), (6, 3, 4)]:
        for x2_shape in [(7, 4), (6, 7, 4)]:
            verify(x1_shape, x2_shape)

@nabenabe0928 nabenabe0928 marked this pull request as ready for review October 14, 2025 07:12
@nabenabe0928 nabenabe0928 added the code-fix Change that does not change the behavior, such as code refactoring. label Oct 14, 2025
@nabenabe0928
Copy link
Copy Markdown
Contributor Author

@sawa3030 @kAIto47802
Could you review this PR?

Copy link
Copy Markdown
Collaborator

@kAIto47802 kAIto47802 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Collaborator

@sawa3030 sawa3030 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nabenabe0928 nabenabe0928 added this to the v4.6.0 milestone Oct 16, 2025
@nabenabe0928 nabenabe0928 disabled auto-merge October 16, 2025 13:25
@y0z y0z disabled auto-merge October 17, 2025 06:22
@sawa3030 sawa3030 merged commit 609924c into optuna:master Oct 17, 2025
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-fix Change that does not change the behavior, such as code refactoring.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants