Ping me when this is ready to be reviewed ;-)

Ready @glemaitre ;)

I have pushed an updated example to use the HistGradientBoosting regressor in the PDP example.

I have also fixed the example to:

• remove the main() function that is no longer needed with loky;
• put the analysis inside the code;
• make the MLPRegressor reach close to .82 R2 score to be comparable with the GBRT model (this requires Quantile based feature scaling + a deeper model + early stopping to avoid long fit times).

Here are the results:

You can observe that the Neural Net PDP now agrees with the GBRT PDB for all the features. It was not the case with a weaker model.

thanks!!

The new plots match Figure 10.16 (p 374) of ESL II even better than the PDP plots of the non-histogram based GBRT.

The test_fastica_simple failure is unrelated. I will open a PR to make it deterministic.

In the plots of the example above, I find it weird that the MLPRegressor partial dependence values are all shifted by +2 w.r.t. the GBRT values. If I switch to method="brute" I also get the shift by +2.

I found the cause of the shift: the offset in y is supposed to happen before the train / test split, otherwise it's either not taken into account or the r2 score cannot be interpreted easily :)

Here is the rendering of the example:

https://58079-843222-gh.circle-artifacts.com/0/doc/auto_examples/inspection/plot_partial_dependence.html#sphx-glr-auto-examples-inspection-plot-partial-dependence-py

One can see that the links to the sklearn.ensemble.HistGradientBoostingRegressor class do not work because of the experimental setup. This is unexpected because the line:

from sklearn.experimental import enable_hist_gradient_boosting  # noqa

is present in doc/conf.py.

The same problem appears in the API table of contents:

https://58079-843222-gh.circle-artifacts.com/0/doc/modules/classes.html#module-sklearn.ensemble

This is probably caused by #13824 that was merged to master concurrently. Let me try to merge master and fix in this PR.

I believe this PR is ready to merge. @glemaitre any further comment?

I find the first plot a bit small:
https://62011-843222-gh.circle-artifacts.com/0/doc/auto_examples/inspection/plot_partial_dependence.html#sphx-glr-auto-examples-inspection-plot-partial-dependence-py

Would it be better to have 2 separate figures?

Otherwise LGTM.

They already are two images. It's sphinx-gallery that's displaying them side by side. But I agree this could be improved by having two code blocks: one for MLPRegressor and one for GBRT. Each figure would appear under the matching code block and that should improve readability.

https://63577-843222-gh.circle-artifacts.com/0/doc/auto_examples/inspection/plot_partial_dependence.html

made 2 code blocks, plots are now bigger

Can you use tight layout or constraint layout to make the ylabels not overlap the plots?

The new one looks slightly better, though it's hard to find something that works both for local and sphinx plots.

The rendering is ok for now. Thanks @NicolasHug