Working on #17671, it appears that GradientBoostingRegressor has an attribute n_classes_ which does not make any sense. We already had something similar in DecisionTreeRegressor some time ago and we deprecate it. I think that we should do something similar here to be sure that we don't have n_classes_ in regressor.
I would almost think that our common test could check this.