An alternative might be too require multioutput to be explicit specified in r2_score

You mean adding a multioutput parameter in RegressorMixin.score. Does this violate our API design?

No, I mean that r2_score would no longer accept being called on multioutput without an explicit choice.

No, I mean that r2_score would no longer accept being called on multioutput without an explicit choice.

You mean we'll raise an error for things like r2_score(y_true, y_pred) when y_type is continuous-multioutput? I doubt whether it's a good idea and it's not related to this PR, right?
(1) If we do so, we'll need to update other regression metrics
(2) If we do so, we'll need to intruduce a multioutput parameter in RegressorMixin.score

Well, it solves this problem since users of r2_score would become aware that the equivalence to .score cannot be taken for granted...?

So with your proposal:
(1) We'll need to update other regression metrics.
(2) Users can't use RegressorMixin.score to evaluate multioutput problems?

And this is not backward compatible, I doubt whether it's worthwhile.

(2) Users can't use RegressorMixin.score to evaluate multioutput problems?

Maybe you mean that we should set multioutput explicitly in RegressorMixin.score?

We already do that??? But we don't document it...

But this is actually a bug introduced in #5143 right?

I still doubt whether it's worthwhile to let all the regression metrics which supports multioutput to raise an error when parameter multioutput is not set explicitly (y_type is multioutput). Is this your final decision?

No I suppose it doesn't really solve the problem... Documentation does.

@jnothman Could you please summarize your proposal here? I'm starting to get confused. Thanks.

I'm surprised to find that we implement the score method in MultiOutputRegressor and are using multioutput='uniform_average' there :)
I prefer to include this is 0.21. @jnothman Is it possible to regard it as a bug fix and change it without a deprecation cycle? If not, what's your proposal here?

Can we stay by documenting the current behaviour?

@jnothman current behavior:
(1)r2_score default multioutput="uniform_average"
(2)MultiOutputRegressor default multioutput="uniform_average"
(3)RegressorMixin (i.e., all other regressors) default multioutput="variance_weighted"
Do you think that's acceptable? (without (2), maybe acceptable)
I think we can regard it as a bug fix because it's erroneously introduced when changing the default value of a parameter.

ready for review @jnothman

ping related people here: @amueller @agramfort @ogrisel
and maybe @adrinjalali

Thanks! Maybe we should tell the user how to avoid this message? It's a bit ugly unfortunately. They can specify it directly when using cv or grid-search.
If they use the score method they would need to import r2 directly.

Thanks! Maybe we should tell the user how to avoid this message? It's a bit ugly unfortunately. They can specify it directly when using cv or grid-search. If they use the score method they would need to import r2 directly.

I've opened #13477 @amueller