What is an issue?

Motivations

The explanation of how to optimize the acquisition function in GPSampler is outdated.

Currently, it is described as Quasi-Monte Carlo (QMC) sampling to optimize the acquisition function. in the document.

However, current implementation to optimize the acquisition function is as follows. See https://github.com/optuna/optuna/blob/master/optuna/_gp/optim_mixed.py for more details.

• Using Quasi Monte-Carlo (QMC) sampling, initial values are sampled and multiple local searches are conducted to return the result with the best acquisition function value. The generation of initial values involves using QMC sampling to select multiple initial values, and then performing a roulette selection from these values to favor those with better acquisition function values.
• Local search is performed using the L-BFGS-B method for continuous variables, linear search for discrete variables, and exhaustive search for categorical variables. Notably, the linear search method applied for discrete variables optimizes a function that returns the linearly interpolated value of the evaluation values of the nearest points on the grid to a given x, and the closest value on the grid to the optimal solution is then returned.
• During the local search, the convergence is determined by continuously checking if the acquisition function value has improved. If no improvement is observed, it is quickly determined that convergence has been reached and the search is terminated.

What should we do?

Update the document of GPSampler to align the current implementation. I think we need to succinctly summarize the key points from the information described above, or actual source codes.