✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 9aded6c. Link to the linter CI: here}

Just wanted to document here that we discussed a potential enhancement for comparing between estimators, where you have cv results from several estimators (so several fold curves for each estimator). Potentially this could be added as a separate function, where you pass the display object, and estimators desired. Not planned, just a potential additional in future.

Hey, I think that you can revive this PR now that the roc curve is merged. Let's try to reuse code from the other PR if possible :)

Thanks @jeremiedbb !

I think @glemaitre mentioned there was some discussion about what to do with the 'chance' level (average precision). In the current PR I have calculated a single average precision (AP) for all the data. I think others suggested that we should calculate average precision for each fold, which I can see is more accurate but I am concerned about the visualization appearance.

Here I have used 5 cv splits and plotting chance for each, and colouring each pair of precision-recall curve/chance line the same colour:

Some concerns about the visualization:

• it would not be unusual for AP to be the same, as has occurred above. The orange and blue lines have the same AP and we can see that results in a single brown-ish line.
• in ROC curve, we decided to colour all CV folds the same colour by default, as we thought this would be most appropriate:
  • as number of folds increases, it would be hard distinguish each line individually
  • we assume we are interested in comparing the results between estimators, thus it makes sense to colour all the cv results from one estimator the same colour

I will have more of a think of a better solution for this.

So for the "Chance level" I would consider all lines to have the same color (and a lower alpha) and in the legend to have a single entry showing the mean + std. I would think it is enough. Also it is easy to link a chance level line with its PR curve because they meet when the recall is 1.0

@jeremiedbb hopefully this is ready for last review now 🙏

Done, thank you @jeremiedbb !

Any more changes @jeremiedbb ? Thank you for review 🙏

@AnneBeyer maybe you may be interested in reviewing this?

This #30508 (comment) hasn't been addressed yet

Thanks all!

Looks like this is breaking the build doc on main because examples/model_selection/plot_cost_sensitive_learning.py uses deprecated kwargs, see build log

  Unexpected failing examples (1):
    
        ../examples/model_selection/plot_cost_sensitive_learning.py failed leaving traceback:
    
        Traceback (most recent call last):
          File "/home/circleci/project/examples/model_selection/plot_cost_sensitive_learning.py", line 366, in <module>
            plot_roc_pr_curves(model, tuned_model, title=title)
          File "/home/circleci/project/examples/model_selection/plot_cost_sensitive_learning.py", line 302, in plot_roc_pr_curves
            PrecisionRecallDisplay.from_estimator(
          File "/home/circleci/project/sklearn/metrics/_plot/precision_recall_curve.py", line 550, in from_estimator
            return cls.from_predictions(
                   ^^^^^^^^^^^^^^^^^^^^^
          File "/home/circleci/project/sklearn/metrics/_plot/precision_recall_curve.py", line 728, in from_predictions
            return viz.plot(
                   ^^^^^^^^^
          File "/home/circleci/project/sklearn/metrics/_plot/precision_recall_curve.py", line 308, in plot
            curve_kwargs = self._validate_curve_kwargs(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/home/circleci/project/sklearn/utils/_plotting.py", line 193, in _validate_curve_kwargs
            warnings.warn(
        FutureWarning: `**kwargs` is deprecated and will be removed in 1.11. Pass all matplotlib arguments to `curve_kwargs` as a dictionary instead.
    
    -------------------------------------------------------------------------------