I am +0.5 on this change. I think the original intention of setting n_estimators is to reduce the runtime of doctest.

No problem, I will amend here and in the corresponding example (#16895 (comment))

I am so sorry for being unclear.

I do prefer removing n_estimators here as long as the increase in doctest runtime is not too great. In this case, the fit runtime is increased by 1~ second, which is acceptable.

Okay I will change it back!

Is there an issue with using load_diabetes here?

I should have mentioned this. Not really but with the diabetes dataset the first 'single' stacking regressor reg gives a R2 score of 0.55 whereas the multiple stacking example (multi_layer_regressor) gives a R2 score of 0.52. Since we state that 'a stacking predictor predicts as good as the best predictor of the base layer' I thought this might be a bad example, or require explaining as to why the score was a little worse? Though the california housing also produced a slightly lower score with the multi_layer_regressor (0.77 vs 0.78).

Hmm for the user guide we can adapt the following:

from sklearn.linear_model import RidgeCV, LassoCV
from sklearn.neighbors import KNeighborsRegressor
estimators = [('ridge', RidgeCV()),
              ('lasso', LassoCV(random_state=42)),
              ('knr', KNeighborsRegressor())]

from sklearn.ensemble import GradientBoostingRegressor
from sklearn.ensemble import StackingRegressor

reg = StackingRegressor(
        estimators=estimators,
        final_estimator=GradientBoostingRegressor(random_state=42)
)

As for the comment about in practice, I think it is suggesting this:

from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

X, y = load_diabetes(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    random_state=42)

predictors = {'stacker': reg}
for name, est in estimators:
    predictors[name] = est

scores = {}
for name, est in predictors.items():
    y_pred = est.fit(X_train, y_train).predict(X_test)
    scores[name] = r2_score(y_test, y_pred)

Which results in:

scores
# {'stacker': 0.5533268189282505,
#  'ridge': 0.49182837290627335,
#  'lasso': 0.4866763449729544,
#  'knr': 0.44659346214225026}

As for the comment about in practice, I think it is suggesting this:

Thanks for clearing that up @thomasjpfan. Would you also expect the result from 'multiple' stacked layers. e.g., multi_layer_regressor here:

final_layer = StackingRegressor(
    estimators=[('rf', RandomForestRegressor(random_state=42)),
                ('gbrt', GradientBoostingRegressor(random_state=42))],
    final_estimator=RidgeCV()
    )
multi_layer_regressor = StackingRegressor(
    estimators=[('ridge', RidgeCV()),
                ('lasso', LassoCV(random_state=42)),
                ('knr', KNeighborsRegressor())],
    final_estimator=final_layer
)

to be similar/at least as good as reg here:

estimators = [('ridge', RidgeCV()),
              ('lasso', LassoCV(random_state=42)),
              ('knr', KNeighborsRegressor())]
reg = StackingRegressor(
    estimators=estimators,
    final_estimator=GradientBoostingRegressor(random_state=42))

or is that more difficult to determine because the final_estimator is different between reg and multi_layer_regressor?

With the diabetes dataset and the switch to KNeighborsRegressor, reg gives a R2 score of 0.55 and multi_layer_regressor gives a R2 score of 0.51.

From my pass experience with stacking, there are times where I see that the stacking does not improve the results as you have demonstrated.

No problem, will amend as you suggest!

Should check for overfitting and tune the model parameters here?

For this specific case, the user guide is trying to demonstrate the stacking API and I would not want to add tuning into the mix. (Specifically, I would not want to introduce GridSearchCV here.)

It would be nice tune the parameters locally and then update the user guide with the tuned parameters.

It would be nice tune the parameters locally and then update the user guide with the tuned parameters.

Sorry that is what I meant! By here I meant for this PR.

Will do thanks for your advice!

@thomasjpfan I've tuned KNeighborsRegressor with diabetes and GradientBoostingRegressor and RandomForestRegressor with the output of the stacked estimators. The R2 value hasn't really changed but it isn't overfitting now.

Thank you! I'm always unsure of formatting in these situations, this was helpful.