[MRG] Fix label_ranking_average_precision_score: sample_weighting isn't applied to items with zero true labels #13412#13447
Conversation
jnothman
left a comment
There was a problem hiding this comment.
Thanks for the pr!!
Please add an entry to the change log at doc/whats_new/v0.21.rst. Like the other entries there, please reference this pull request with :issue: and credit yourself (and other contributors if applicable) with :user:
| n_classes, n_samples, random_state) | ||
|
|
||
|
|
||
| def test_lrap_sample_weighting(): |
There was a problem hiding this comment.
This is tested by test_common.py unless I'm much mistaken. However it seems we don't generate any samples without labels there, perhaps intentionally since I think some metrics might not handle it
| assert_almost_equal( | ||
| label_ranking_average_precision_score(y_true, y_score, | ||
| sample_weight=sample_weight), | ||
| np.sum(sample_weight * samplewise_lraps)/np.sum(sample_weight)) |
|
Thanks for the feedback; all fixed I believe. PTAL. |
doc/whats_new/v0.21.rst
Outdated
|
|
||
| - |Fix| :func:`metrics.label_ranking_average_precision_score` was returning | ||
| incorrect values where some samples had degenerate labels and `sample_weight` | ||
| was used. |
There was a problem hiding this comment.
It seems that you're saying that sample_weight shouldn't be used.
How about:
Fixed a bug in [...] where
sample_weightwasn't taken into account for samples with degenerate labels
There was a problem hiding this comment.
ping @dpwe ;) could you please address this so we can merge?
There was a problem hiding this comment.
Sorry for the delay. I made the change you suggested.
|
Thanks @dpwe! |
…cikit-learn#13447)" This reverts commit 456055d.
…cikit-learn#13447)" This reverts commit 456055d.
Fixes #13412
What does this implement/fix? Explain your changes.
sklearn.metrics.label_ranking_average_precision_scoresupports asample_weightargument to allow different contributions of individual trial samples. However, in the case where certain samples had degenerate labels (all true or none true), the special case logic that gave these samples a precision of 1.0 failed to apply thesample_weights. This is now corrected, with tests that exercisesample_weightwithout degenerate labels (a test that passed before the code was modified), and with degenerate labels (a test that failed until this change was applied).Any other comments?