Skip to content

Make the interface of batched_lbfgsb module compatible with scipy.optimize#6273

Merged
nabenabe0928 merged 52 commits intooptuna:masterfrom
Kaichi-Irie:refactor-batched-lbfgsb-for-PR
Oct 17, 2025
Merged

Make the interface of batched_lbfgsb module compatible with scipy.optimize#6273
nabenabe0928 merged 52 commits intooptuna:masterfrom
Kaichi-Irie:refactor-batched-lbfgsb-for-PR

Conversation

@Kaichi-Irie
Copy link
Copy Markdown
Contributor

@Kaichi-Irie Kaichi-Irie commented Sep 12, 2025

Motivation

This PR introduces a refactoring of batched_lbfgsb and its related functions, making the interface of the batched_lbfgsb compatible with the scipy.optimize module. The goal is to address two pieces of technical debt that could lead to future bugs and make the API less intuitive.

  1. Resolve Variable Capture to Prevent Bugs
    The core issue is that the nested function negative_acqf_with_grad was implicitly using the normalized_params variable from its enclosing scope (_gradient_ascent_batched). While this may not be causing an active bug, it is a form of technical debt that makes the code harder to reason about and could easily lead to errors in the future. This PR resolves the capture.

  2. Improve API Flexibility for func_and_grad
    The current implementation of batched_lbfgsb implicitly requires that the func_and_grad callable accepts a second argument (batch_indices), even if the function itself does not use it. This design is inflexible and counter-intuitive.

For example, a simple single-argument function is currently incompatible:

# This function is not used, but is required as a placeholder argument
def myfunc1(x: np.ndarray, batched_indices: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
    value = -np.sum(x**2)
    grad = -2 * x
    return value, grad

# This more natural, single-argument function should work, but currently throws an error
def myfunc2(x: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
    value = -np.sum(x**2)
    grad = -2 * x
    return value, grad

# Works correctly
batched_lbfgsb(myfunc1, ...)

# Fails with a TypeError because the second argument is missing
batched_lbfgsb(myfunc2, ...)

This change allows batched_lbfgsb to correctly handle callables with and without the second argument, making the API more flexible and user-friendly.

Description of the changes

  • The function signature for batched_lbfgsb has been updated.
  • The batch_indices argument has been removed fromfunc_and_grad. This information is now passed via args_list instead.

@Kaichi-Irie Kaichi-Irie changed the title Refactor batched lbfgsb for pr Make the interface of batched lbfgsb compatible with SciPy optimize Sep 12, 2025
@nabenabe0928 nabenabe0928 added the code-fix Change that does not change the behavior, such as code refactoring. label Sep 12, 2025
@nabenabe0928 nabenabe0928 added this to the v4.6.0 milestone Sep 12, 2025
@Kaichi-Irie Kaichi-Irie changed the title Make the interface of batched lbfgsb compatible with SciPy optimize Make the interface of batched_lbfgsb module compatible with scipy.optimize Sep 13, 2025
@nabenabe0928
Copy link
Copy Markdown
Contributor

@not522
Could you review this PR?

@nabenabe0928
Copy link
Copy Markdown
Contributor

@not522 Thank you for the review, but as we discussed internally, I will re-assign to another reviewer next week...!

Copy link
Copy Markdown
Contributor

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kaichi-Irie
Could you merge the master branch and modify the corresponding unit tests accordingly?

Copy link
Copy Markdown
Contributor

@nabenabe0928 nabenabe0928 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix, LGTM!

@nabenabe0928 nabenabe0928 merged commit a07c426 into optuna:master Oct 17, 2025
11 of 12 checks passed
@Kaichi-Irie Kaichi-Irie deleted the refactor-batched-lbfgsb-for-PR branch October 17, 2025 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-fix Change that does not change the behavior, such as code refactoring.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants