DOC Update plot_permutation_test_for_classification.py by lucyleeow · Pull Request #17385 · scikit-learn/scikit-learn

lucyleeow · 2020-05-29T14:25:17Z

Reference Issues/PRs

What does this implement/fix? Explain your changes.

use notebook style alternating code and text blocks
amend example to use calculate score with both structured and unstructured data
Expand explanations

Any other comments?

thomasjpfan

Thank you @lucyleeow !

thomasjpfan · 2020-06-01T21:27:06Z

examples/feature_selection/plot_permutation_test_for_classification.py

+#
+# Next, we calculate the
+# :func:`~sklearn.model_selection.permutation_test_score` using the original
+# iris dataset, which has strong structure, and


Maybe:

Suggested change

# iris dataset, which has strong structure, and

# iris dataset, which strongly predict the labels, and

thomasjpfan · 2020-06-01T21:27:44Z

examples/feature_selection/plot_permutation_test_for_classification.py

+# iris dataset, which has strong structure, and
+# the randomly generated features and iris labels, which should have
+# no dependency between features and labels. The
+# :class:`~sklearn.svm.svc` classifier and :ref:`accuracy_score` are used.


Suggested change

# :class:`~sklearn.svm.svc` classifier and :ref:`accuracy_score` are used.

# :class:`~sklearn.svm.SVC` classifier and :ref:`accuracy_score` are used.

thomasjpfan · 2020-06-01T21:29:11Z

examples/feature_selection/plot_permutation_test_for_classification.py

+# distribution by calculating the accuracy of the classifier
+# on 1000 different permutations of the dataset, where features
+# remain the same but labels undergo different permutations. This is the
+# distribution for the null hypothesis that there is no dependency between


Suggested change

# distribution for the null hypothesis that there is no dependency between

# distribution for the null hypothesis which states there is no dependency between

thomasjpfan · 2020-06-01T21:31:49Z

examples/feature_selection/plot_permutation_test_for_classification.py

-plt.legend()
-plt.xlabel('Score')
+score_iris, perm_scores_iris, pvalue_iris = permutation_test_score(
+    clf, X, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)


Default is 1:

Suggested change

clf, X, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)

clf, X, y, scoring="accuracy", cv=cv, n_permutations=1000)

thomasjpfan · 2020-06-01T21:31:56Z

examples/feature_selection/plot_permutation_test_for_classification.py

+    clf, X, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)
+
+score_rand, perm_scores_rand, pvalue_rand = permutation_test_score(
+    clf, X_rand, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)


Suggested change

clf, X_rand, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)

clf, X_rand, y, scoring="accuracy", cv=cv, n_permutations=1000)

lucyleeow · 2020-06-02T08:43:07Z

Thanks for the review @thomasjpfan! Amended.

thomasjpfan

LGTM

glemaitre

Overall looks good. Just little improvements to be considered.

examples/feature_selection/plot_permutation_test_for_classification.py

lucyleeow · 2020-06-18T10:08:46Z

Thanks @glemaitre! Just let me know what you think we should do to avoid the text output from matplotlib: #17385 (comment)

lucyleeow · 2020-07-16T10:21:06Z

ping @glemaitre, changes made.

…ot_perm

ogrisel

LGTM.

…17385)

lucyleeow added 8 commits May 29, 2020 16:05

amend

c9f377f

expand

4244871

fix var name

a540513

fix var name

2597b74

format

02d1a1c

add auth

e3256b3

add auth

b260bdb

set rand

3c2c483

lucyleeow mentioned this pull request May 29, 2020

DOC Add doc for permutation_test_score #17373

Merged

lucyleeow added 2 commits May 29, 2020 17:43

fix ref

7620edc

fix ref

a3e7c8c

lucyleeow changed the title ~~WIP DOC Update plot_permutation_test_for_classification.py~~ DOC Update plot_permutation_test_for_classification.py May 29, 2020

thomasjpfan reviewed Jun 1, 2020

View reviewed changes

suggestions

dd154e0

thomasjpfan approved these changes Jun 2, 2020

View reviewed changes

cmarmo added the Waiting for Reviewer label Jun 14, 2020

glemaitre removed the Waiting for Reviewer label Jun 18, 2020

glemaitre self-requested a review June 18, 2020 09:31

glemaitre reviewed Jun 18, 2020

View reviewed changes

suggestions

f46f474

lucyleeow added 2 commits June 18, 2020 12:09

lint

d38e85a

plt

fb63b68

ogrisel added 2 commits July 31, 2020 10:45

Trigger CI to rebuild doc artifacts

468e1c6

Merge branch 'master' of github.com:scikit-learn/scikit-learn into pl…

d1d2400

…ot_perm

ogrisel approved these changes Jul 31, 2020

View reviewed changes

ogrisel merged commit e087f8a into scikit-learn:master Jul 31, 2020

lucyleeow deleted the plot_perm branch July 31, 2020 20:10

jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020

DOC Update plot_permutation_test_for_classification.py (scikit-learn#…

1c6618e

…17385)

	# iris dataset, which has strong structure, and
	# iris dataset, which strongly predict the labels, and

	# :class:`~sklearn.svm.svc` classifier and :ref:`accuracy_score` are used.
	# :class:`~sklearn.svm.SVC` classifier and :ref:`accuracy_score` are used.

	# distribution for the null hypothesis that there is no dependency between
	# distribution for the null hypothesis which states there is no dependency between

	clf, X, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)
	clf, X, y, scoring="accuracy", cv=cv, n_permutations=1000)

	clf, X_rand, y, scoring="accuracy", cv=cv, n_permutations=1000, n_jobs=1)
	clf, X_rand, y, scoring="accuracy", cv=cv, n_permutations=1000)

Uh oh!

Conversation

lucyleeow commented May 29, 2020

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

Uh oh!

thomasjpfan left a comment

Choose a reason for hiding this comment

Uh oh!

thomasjpfan Jun 1, 2020

Choose a reason for hiding this comment

Uh oh!

thomasjpfan Jun 1, 2020

Choose a reason for hiding this comment

Uh oh!

thomasjpfan Jun 1, 2020

Choose a reason for hiding this comment

Uh oh!

thomasjpfan Jun 1, 2020

Choose a reason for hiding this comment

Uh oh!

thomasjpfan Jun 1, 2020

Choose a reason for hiding this comment

Uh oh!

lucyleeow commented Jun 2, 2020

Uh oh!

thomasjpfan left a comment

Choose a reason for hiding this comment

Uh oh!

glemaitre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lucyleeow commented Jun 18, 2020

Uh oh!

lucyleeow commented Jul 16, 2020

Uh oh!

ogrisel left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants