-
-
Notifications
You must be signed in to change notification settings - Fork 26.9k
Add PAV algorithm for calibration_curve/reliability diagrams #23132
Description
Describe the workflow you want to enable
import numpy as np
from sklearn.calibration import calibration_curve
y_true = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1])
y_pred = np.array([0.1, 0.2, 0.3, 0.4, 0.65, 0.7, 0.8, 0.9, 1.])
prob_true, prob_pred = calibration_curve(y_true, y_pred, strategy="pav")Describe your proposed solution
Add the strategy PAV as [1] and [2] to calibration_curve, there called CORP. This is basically applying isotonic regression as binning strategy, which we already have in scikit-learn.
[1] Dimitriadis, T., Gneiting, T., & Jordan, A.I. (2021). Stable reliability diagrams for probabilistic classifiers. Proceedings of the National Academy of Sciences of the United States of America, 118. https://doi.org/10.1073/pnas.2016191118
[2] https://cran.r-project.org/package=reliabilitydiag
Describe alternatives you've considered, if relevant
No response
Additional context
Given the recency of the paper, it has clearly not enough citations (yet). But I have the impression that this is a good strategy for a reliability diagrams with good theoretical and practical properties.
To my knowledge, this strategy is nowhere available in the Python universe as of now.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status