Common check for sample weight invariance with removed samples by rth · Pull Request #17176 · scikit-learn/scikit-learn

rth · 2020-05-10T15:11:56Z

Same as #16507 but taking into account latest updates in check_estimator and handling of xfail estimators.

I initially tried to fix this quickly in #17175, but in the end decided to revert the inital commit to fix CI and do a less rushed solution in this PR.

…t-learn#16507) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>

rth · 2020-05-10T15:17:25Z

cc @NicolasHug .

We need to first merge #16963 I think. Earlier version of this PR basically implemented a simpler version of that PR.

NicolasHug · 2020-05-10T15:19:18Z

I guess we need to start with #17134 then :p

…-checks-updates

rth · 2020-05-15T18:14:42Z

@NicolasHug This PR should be good now. The coverage failure doesn't look relevant.

NicolasHug

Thanks @rth, a few nits but LGTM!

NicolasHug · 2020-05-15T18:54:26Z

sklearn/utils/estimator_checks.py

+def check_sample_weights_invariance(name, estimator_orig, kind="ones"):
    # check that the estimators yield same results for
    # unit weights and no weights
    if (has_fit_parameter(estimator_orig, "sample_weight") and


while we're at it we might as well return early here instead of having the entire function in a if block?

Actually moved this _yield_all_checks to only run check_sample_weight on estimators that support it. If find that showing a green status for a check when it doesn't apply is not great (and skipping it also produces too much noise).

sklearn/utils/estimator_checks.py

NicolasHug · 2020-05-15T19:07:34Z

sklearn/utils/estimator_checks.py

                      [2, 1], [2, 1], [2, 1], [2, 1],
                      [3, 3], [3, 3], [3, 3], [3, 3],
-                      [4, 1], [4, 1], [4, 1], [4, 1]], dtype=np.dtype('float'))
+                      [4, 1], [4, 1], [4, 1], [4, 1]], dtype=np.float64)


for consistency these could be X1 and y1

Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

thomasjpfan

Thank you @rth

thomasjpfan · 2020-05-16T23:02:02Z

sklearn/utils/estimator_checks.py

+            if sparse.issparse(X_pred1):
+                X_pred1 = X_pred1.toarray()
+                X_pred2 = X_pred2.toarray()
+            assert_allclose(X_pred1, X_pred2, err_msg=err_msg)


We can use sklearn.utils._testing.assert_allclose_dense_sparse here.

…-checks-updates

…t-learn#17176) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>

…ple is never a support vector

…t-learn#17176) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>

…ple is never a support vector

rth and others added 4 commits May 10, 2020 17:09

Common check for sample weight invariance with removed samples (sciki…

3deca92

…t-learn#16507) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>

_xfail_tests -> _xfail_checks & revert changes to check_estimator

8988e1f

Fix docstring format

2b0bf2c

More docstring fixes

ecec873

rth mentioned this pull request May 10, 2020

TST Add tests for LinearRegression that sample weights act consistently #15554

Merged

Merge remote-tracking branch 'upstream/master' into inveriance-common…

4c7ffbd

…-checks-updates

NicolasHug approved these changes May 15, 2020

View reviewed changes

rth and others added 3 commits May 15, 2020 21:24

Update sklearn/utils/estimator_checks.py

bbc0918

Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

Update sklearn/utils/estimator_checks.py

efc8582

Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>

Address Nicolas' comments

5a74380

thomasjpfan approved these changes May 16, 2020

View reviewed changes

rth added 2 commits June 3, 2020 22:34

Merge remote-tracking branch 'upstream/master' into inveriance-common…

ece8426

…-checks-updates

Address review comments

f481bcc

rth changed the title ~~Common check for sample weight invariance with removed samples (updated)~~ Common check for sample weight invariance with removed samples Jun 3, 2020

rth merged commit 8feb045 into scikit-learn:master Jun 3, 2020

rth deleted the inveriance-common-checks-updates branch June 3, 2020 22:30

viclafargue pushed a commit to viclafargue/scikit-learn that referenced this pull request Jun 26, 2020

Common check for sample weight invariance with removed samples (sciki…

4d27f38

…t-learn#17176) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 5, 2020

reflect scikit-learn#17176: zero weight in SV models means that a sam…

5847ec3

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 5, 2020

reflect scikit-learn#17176: zero weight in SV models means that a sam…

80e4f84

…ple is never a support vector

rth mentioned this pull request Jul 25, 2020

Add Hypothesis Testing #13846

Open

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Aug 5, 2020

reflect scikit-learn#17176: zero weight in SV models means that a sam…

1da7c0e

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Oct 15, 2020

reflect scikit-learn#17176: zero weight in SV models means that a sam…

6d68af7

…ple is never a support vector

jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020

Common check for sample weight invariance with removed samples (sciki…

c60c2a5

…t-learn#17176) Co-Authored-By: Nicolas Hug <contact@nicolas-hug.com>

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Nov 28, 2020

reflect scikit-learn#17176: zero weight in SV models means that a sam…

89446bf

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Dec 12, 2020

reflect scikit-learn#17176: zero weight in SV models means that a sam…

89bb3cf

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Feb 25, 2021

reflect scikit-learn#17176: zero weight in SV models means that a sam…

da5d34d

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Feb 25, 2021

reflect scikit-learn#17176: zero weight in SV models means that a sam…

a261314

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Mar 27, 2021

reflect scikit-learn#17176: zero weight in SV models means that a sam…

895e885

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jun 15, 2021

reflect scikit-learn#17176: zero weight in SV models means that a sam…

db4f78e

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 23, 2021

reflect scikit-learn#17176: zero weight in SV models means that a sam…

67c3093

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Nov 10, 2021

reflect scikit-learn#17176: zero weight in SV models means that a sam…

8597460

…ple is never a support vector

This was referenced Dec 9, 2021

Test sample_weight in common tests #5367

Closed

RFC Sample weight invariance properties #15657

Open

ivannz added a commit to ivannz/scikit-learn that referenced this pull request May 15, 2022

reflect scikit-learn#17176: zero weight in SV models means that a sam…

d0a6d28

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jun 14, 2022

reflect scikit-learn#17176: zero weight in SV models means that a sam…

a746e06

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Aug 29, 2022

reflect scikit-learn#17176: zero weight in SV models means that a sam…

c88c162

…ple is never a support vector

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Sep 5, 2022

reflect scikit-learn#17176: zero weight in SV models means that a sam…

7a0ede0

…ple is never a support vector

lorentzenchr mentioned this pull request Oct 25, 2024

Refactor tests for sample weights #11316

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Common check for sample weight invariance with removed samples#17176

Common check for sample weight invariance with removed samples#17176
rth merged 10 commits intoscikit-learn:masterfrom
rth:inveriance-common-checks-updates

rth commented May 10, 2020 •

edited

Loading

Uh oh!

rth commented May 10, 2020

Uh oh!

NicolasHug commented May 10, 2020

Uh oh!

rth commented May 15, 2020

Uh oh!

NicolasHug left a comment

Uh oh!

NicolasHug May 15, 2020

Uh oh!

rth May 15, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug May 15, 2020

Uh oh!

thomasjpfan left a comment

Uh oh!

thomasjpfan May 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

rth commented May 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rth commented May 10, 2020

Uh oh!

NicolasHug commented May 10, 2020

Uh oh!

rth commented May 15, 2020

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

NicolasHug May 15, 2020

Choose a reason for hiding this comment

Uh oh!

rth May 15, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug May 15, 2020

Choose a reason for hiding this comment

Uh oh!

thomasjpfan left a comment

Choose a reason for hiding this comment

Uh oh!

thomasjpfan May 16, 2020

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rth commented May 10, 2020 •

edited

Loading