FIX Correct the definition of gamma=`scale` in svm by qinhanmin2014 · Pull Request #13221 · scikit-learn/scikit-learn

qinhanmin2014 · 2019-02-22T11:41:40Z

Closes #12741
See #13186 (comment):
And I'm wondering whether it's possible to solve #12741 in 0.20.3 by changing the definition of gamma='scale' directly, since it's introduced erroneously in 0.20. This is an embarrassing mistake and I think it'll be much more difficult to solve it in 0.21.X (maybe at that time we'll need to deprecate scale and introduce another option).

qinhanmin2014 · 2019-02-22T13:37:59Z

sklearn/model_selection/tests/test_search.py

                    "to False in version 0.22")
    X, y = make_blobs(n_samples=54, random_state=0, centers=2)
-    grid = GridSearchCV(SVC(gamma='scale'), param_grid={'C': [1]}, cv=3)
+    grid = GridSearchCV(SVC(gamma='scale', random_state=0),


We do not always warn.

scikit-learn/sklearn/model_selection/_search.py

Lines 795 to 813 in d19a5dc

if self.iid == 'warn':

warn = False

for scorer_name in scorers.keys():

scores = test_scores[scorer_name].reshape(n_candidates,

n_splits)

means_weighted = np.average(scores, axis=1,

weights=test_sample_counts)

means_unweighted = np.average(scores, axis=1)

if not np.allclose(means_weighted, means_unweighted,

rtol=1e-4, atol=1e-4):

warn = True

break

if warn:

warnings.warn("The default of the `iid` parameter will change "

"from True to False in version 0.22 and will be"

" removed in 0.24. This will change numeric"

" results when test-set sizes are unequal.",

DeprecationWarning)

qinhanmin2014 · 2019-02-22T13:38:07Z

sklearn/svm/tests/test_sparse.py

    for dataset in datasets:
        for kernel in kernels:
-            clf = svm.SVC(gamma='scale', kernel=kernel, probability=True,
+            clf = svm.SVC(gamma=1, kernel=kernel, probability=True,


numerical issue here and below

X = np.array([[-2, -1], [-1, -1], [-1, -2], [1, 1], [1, 2], [2, 1]]) X_sp = sparse.lil_matrix(X) Y = [1, 1, 1, 2, 2, 2] clf = svm.SVC(gamma='scale', kernel=kernel, probability=True, random_state=0, decision_function_shape='ovo') sp_clf = svm.SVC(gamma='scale', kernel=kernel, probability=True, random_state=0, decision_function_shape='ovo') clf.fit(X, Y) print(clf._gamma) print(clf.support_) sp_clf.fit(X_sp, Y) print(sp_clf._gamma) print(sp_clf.support_) # 0.25 # [0 1 3 4] # 0.25000000000000006 # [1 2 3 5]

qinhanmin2014 · 2019-02-22T14:03:43Z

doc/tutorial/basic/tutorial.rst

    tol=0.001, verbose=False)
  >>> clf.predict(X_test)
-  array([1, 0, 1, 1, 0])
+  array([0, 0, 0, 1, 0])


these samples are generated randomly (maybe we should avoid doing so in the tutorial?)

qinhanmin2014 · 2019-02-22T14:40:22Z

Ready for a review @jnothman @amueller
We need to change some tests, see my reviews above.
Tagging 0.20.3.

ogrisel

LGTM. But because gamma is now a fitted parameter, wouldn't it make sense to expose the estimated bandwidth as a public attribute gamma_ instead of a private attribute _gamma?

I would be +0 for making it public.

amueller · 2019-02-22T17:33:22Z

lgtm (but github doesn't let me click approve?)

qinhanmin2014 · 2019-02-23T01:04:21Z

these samples are generated randomly (maybe we should avoid doing so in the tutorial?)

Will soon open a PR to update the tutorial.

But because gamma is now a fitted parameter, wouldn't it make sense to expose the estimated bandwidth as a public attribute gamma_ instead of a private attribute _gamma?

Also +0 from my side and this is out of the scope of this PR. Maybe open an issue/PR if someone wants it?

…3221

…arn#13221)" This reverts commit 7d8830d.

…3221

qinhanmin2014 added 5 commits February 22, 2019 14:53

svm gamma=auto

f1ecc5f

test fix

99c57a3

test

a489a95

test

c579d8d

test

d7f0abe

qinhanmin2014 commented Feb 22, 2019

View reviewed changes

test

819da9e

qinhanmin2014 commented Feb 22, 2019

View reviewed changes

what's new

0314703

qinhanmin2014 added this to the 0.20.3 milestone Feb 22, 2019

ogrisel approved these changes Feb 22, 2019

View reviewed changes

jnothman approved these changes Feb 24, 2019

View reviewed changes

jnothman merged commit 5c8167a into scikit-learn:master Feb 24, 2019

jnothman pushed a commit to jnothman/scikit-learn that referenced this pull request Feb 24, 2019

FIX Correct the definition of gamma=scale in svm (scikit-learn#13221)

b098eb5

qinhanmin2014 deleted the svm-gamma branch February 24, 2019 07:44

qinhanmin2014 mentioned this pull request Feb 24, 2019

DOC Avoid training/predicting on random dataset in the tutorial #13234

Merged

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Mar 19, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

ea266ce

…3221

ivannz mentioned this pull request Mar 19, 2019

[MRG+1] Support Vector Data Description #7910

Closed

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Apr 23, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

a654f2c

…3221

xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019

FIX Correct the definition of gamma=scale in svm (scikit-learn#13221)

7d8830d

xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019

Revert "FIX Correct the definition of gamma=scale in svm (scikit-le…

8b6121b

…arn#13221)" This reverts commit 7d8830d.

xhluca pushed a commit to xhluca/scikit-learn that referenced this pull request Apr 28, 2019

Revert "FIX Correct the definition of gamma=scale in svm (scikit-le…

e7dee80

…arn#13221)" This reverts commit 7d8830d.

koenvandevelde pushed a commit to koenvandevelde/scikit-learn that referenced this pull request Jul 12, 2019

FIX Correct the definition of gamma=scale in svm (scikit-learn#13221)

ec032bb

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 25, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

5f3aaee

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Aug 22, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

25b6b47

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Oct 5, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

2943d66

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Oct 11, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

b2eb6ad

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Nov 1, 2019

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

7e3b974

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jan 10, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

86178c6

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jan 30, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

4982cab

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jan 30, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

f9392b6

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Feb 27, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

e5d8b25

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Mar 12, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

ca3f0a0

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Apr 17, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

a9fd428

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request May 3, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

affc8b4

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request May 30, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

71f8d77

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 5, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

031088c

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 5, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

91d4b55

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Aug 5, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

fd841de

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Oct 15, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

a49a602

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Nov 28, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

54bfbb0

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Dec 12, 2020

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

ec6f2f5

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Feb 25, 2021

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

c2606fb

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Feb 25, 2021

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

595be5d

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Mar 27, 2021

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

52c4a53

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jun 15, 2021

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

0d4b3e1

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jul 23, 2021

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

639ac1e

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Nov 10, 2021

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

c2944a8

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request May 15, 2022

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

bd5fe04

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Jun 14, 2022

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

bcfa94f

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Aug 29, 2022

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

7a1f3d1

…3221

ivannz added a commit to ivannz/scikit-learn that referenced this pull request Sep 5, 2022

TST Fixed SVDD tests affected by scale redefinition in scikit-learn#1…

355c548

…3221

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FIX Correct the definition of gamma=`scale` in svm#13221

FIX Correct the definition of gamma=`scale` in svm#13221
jnothman merged 7 commits intoscikit-learn:masterfrom
qinhanmin2014:svm-gamma

qinhanmin2014 commented Feb 22, 2019

Uh oh!

qinhanmin2014 Feb 22, 2019

Uh oh!

qinhanmin2014 Feb 22, 2019

Uh oh!

qinhanmin2014 Feb 22, 2019

Uh oh!

qinhanmin2014 commented Feb 22, 2019

Uh oh!

ogrisel left a comment

Uh oh!

amueller commented Feb 22, 2019

Uh oh!

qinhanmin2014 commented Feb 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	if self.iid == 'warn':
	warn = False
	for scorer_name in scorers.keys():
	scores = test_scores[scorer_name].reshape(n_candidates,
	n_splits)
	means_weighted = np.average(scores, axis=1,
	weights=test_sample_counts)
	means_unweighted = np.average(scores, axis=1)
	if not np.allclose(means_weighted, means_unweighted,
	rtol=1e-4, atol=1e-4):
	warn = True
	break

	if warn:
	warnings.warn("The default of the `iid` parameter will change "
	"from True to False in version 0.22 and will be"
	" removed in 0.24. This will change numeric"
	" results when test-set sizes are unequal.",
	DeprecationWarning)

Uh oh!

Conversation

qinhanmin2014 commented Feb 22, 2019

Uh oh!

qinhanmin2014 Feb 22, 2019

Choose a reason for hiding this comment

Uh oh!

qinhanmin2014 Feb 22, 2019

Choose a reason for hiding this comment

Uh oh!

qinhanmin2014 Feb 22, 2019

Choose a reason for hiding this comment

Uh oh!

qinhanmin2014 commented Feb 22, 2019

Uh oh!

ogrisel left a comment

Choose a reason for hiding this comment

Uh oh!

amueller commented Feb 22, 2019

Uh oh!

qinhanmin2014 commented Feb 23, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants