Currently, the following list of metrics will raise an error if str are provided as labels and that pos_label is not defined and needed to compute the metrics:

• average_precision_score
• f1_score
• fbeta_score
• jaccard_score
• precision_recall_curve
• precision_score
• recall_score
• roc_curve

We try to make sure that the error message is consistent in #18192

brier_score_loss should supposedly follow the behaviour than the other metrics. However, it seems that up-to-now, it does some inference when labels are string.

Thus, 2 questions can be raised:

• Is it normal that brier_score_loss can do such inference or should it raise a similar error as other metrics?
• If it should raise an error, shall we open consider as a bug fix or make a deprecation cycle?