[MRG] Updates to SVM User Guide by NicolasHug · Pull Request #16769 · scikit-learn/scikit-learn

NicolasHug · 2020-03-25T23:59:52Z

add references throughout the text
put examples in their respective sections
update docstring examples to use a StandardScaler
add detail about shrinking param. I don't know how to detail tol and max_iter though (seems really libsvm-specific).
add short descriptions to figures
add details in math section and reference the hinge and eps-insensitive losses
a few fixes / clarifications here and there

…m_UG

NicolasHug · 2020-03-26T18:32:33Z

doc/modules/svm.rst

    >>> rbf_svc.kernel
    'rbf'

+Parameters of the RBF Kernel


I just moved that up

NicolasHug · 2020-03-26T18:34:09Z

pinging the suggested reviewers @thomasjpfan @glemaitre @qinhanmin2014

adrinjalali

Thanks @NicolasHug , love having better user guides :)

will try to review the rest tomorrow :)

adrinjalali · 2020-03-26T12:44:48Z

doc/modules/svm.rst

-multi-class strategy, thus training n_class models. If there are only
-two classes, only one model is trained::


any particular reason to remove the sentence here?

The whole section is about "multiclass classification"

Also the example below is an illustration of multiclass, not binary as the sentence suggests, so it's misleading

doc/modules/svm.rst

adrinjalali · 2020-03-26T19:01:07Z

doc/modules/svm.rst

-:class:`LinearSVC` by the `liblinear`_ implementation is much more
-efficient than its `libsvm`_-based :class:`SVC` counterpart and can


don't the liblinear and libsvm actually point to the right places?

adrinjalali · 2020-03-26T19:10:10Z

doc/modules/svm.rst

-generalization error of the classifier.
-
+generalization error of the classifier. The figure below shows the decision
+function for a separable problem, with three samples within the margin


Suggested change

function for a separable problem, with three samples within the margin

function for a linearly separable problem, with three samples on the margin

maybe?

indeed these are on the margin but in general, i.e. when classes aren't separable, the SVs are "within" the margin boundary

doc/modules/svm.rst

adrinjalali · 2020-03-26T19:17:29Z

doc/modules/svm.rst

+is the kernel. The terms :math:`\alpha_i` are called the dual coefficients.
+This dual representation highlights the fact that training vectors are
+implicitly mapped into a higher (maybe infinite)
+dimensional space by the function :math:`\phi`.


It's really tricky to explain why the dual highlights this fact. Hmm

"tricky"

I see what you did there.

(all jokes aside yeah, we can't really explain that unless we explain the kernel trick. I'll add a link to https://en.wikipedia.org/wiki/Kernel_method which is reasonably good)

adrinjalali · 2020-03-26T19:21:18Z

doc/modules/svm.rst

-    While SVM models derived from `libsvm`_ and `liblinear`_ use ``C`` as
+    While SVM models derived from `libsvm` and `liblinear` use ``C`` as


same here with links.

Co-Authored-By: Adrin Jalali <adrin.jalali@gmail.com>

NicolasHug

Thanks for the fast review!

NicolasHug · 2020-03-26T19:25:24Z

doc/modules/svm.rst

-multi-class strategy, thus training n_class models. If there are only
-two classes, only one model is trained::


The whole section is about "multiclass classification"

Also the example below is an illustration of multiclass, not binary as the sentence suggests, so it's misleading

NicolasHug · 2020-03-26T19:28:34Z

doc/modules/svm.rst

-generalization error of the classifier.
-
+generalization error of the classifier. The figure below shows the decision
+function for a separable problem, with three samples within the margin


indeed these are on the margin but in general, i.e. when classes aren't separable, the SVs are "within" the margin boundary

NicolasHug · 2020-03-26T19:31:01Z

doc/modules/svm.rst

+is the kernel. The terms :math:`\alpha_i` are called the dual coefficients.
+This dual representation highlights the fact that training vectors are
+implicitly mapped into a higher (maybe infinite)
+dimensional space by the function :math:`\phi`.


"tricky"

I see what you did there.

(all jokes aside yeah, we can't really explain that unless we explain the kernel trick. I'll add a link to https://en.wikipedia.org/wiki/Kernel_method which is reasonably good)

doc/modules/svm.rst

thomasjpfan

Changing the notation from rho to b and including scaling in the examples are great.

Minor comments, otherwise LGTM.

doc/modules/svm.rst

adrinjalali · 2020-04-06T09:46:41Z

doc/modules/svm.rst

 (maybe infinite) dimensional space by the function :math:`\phi`.

-The decision function is:
+The prediction is:


But it is the prediction is the sign of this, isn't it?

That's for regression (I did fix it for classification)

sklearn/svm/_classes.py

…m_UG

* WIP * WIP * Updates to the math section * so apparently crammer-singer isn't consistent (http://www.cs.columbia.edu/\~rocco/papers/icml13.html) * reorganized example links and described pictures * Added references * more refs * Added scaler to docstring examples * doc about dual coefs * WIP * note about shrinking and gram matrix * ellipsis * ellipsis again * maybe fixed doc issue * details for regression * training error -> margin error * Apply suggestions from code review Co-Authored-By: Adrin Jalali <adrin.jalali@gmail.com> * put back links + addressed rest of comments * small detail about margin boundaries and SVs * Addressed comments from Thomas and Adrin Co-authored-by: Adrin Jalali <adrin.jalali@gmail.com>

NicolasHug added 12 commits March 20, 2020 21:25

WIP

3c4601a

WIP

39e8725

Updates to the math section

d23862f

so apparently crammer-singer isn't consistent (http://www.cs.columbia…

ffa63cd

….edu/\~rocco/papers/icml13.html)

reorganized example links and described pictures

3ed4798

Added references

de38051

more refs

254ee14

Added scaler to docstring examples

bca657a

doc about dual coefs

0617fba

WIP

a0d9671

Merge branch 'master' of github.com:scikit-learn/scikit-learn into sv…

0f1a0c2

…m_UG

note about shrinking and gram matrix

f4095b0

github-actions bot added the module:svm label Mar 26, 2020

NicolasHug added 5 commits March 26, 2020 08:23

ellipsis

d792f2c

ellipsis again

a589ec1

maybe fixed doc issue

16f689b

details for regression

0bdd595

training error -> margin error

4d11886

NicolasHug commented Mar 26, 2020

View reviewed changes

doc/modules/svm.rst

>>> rbf_svc.kernel

'rbf'

Parameters of the RBF Kernel

Copy link
Copy Markdown

Member Author

NicolasHug Mar 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved that up

NicolasHug changed the title ~~[WIP] Updates to SVM User Guide~~ [MRG] Updates to SVM User Guide Mar 26, 2020

adrinjalali reviewed Mar 26, 2020

View reviewed changes

NicolasHug and others added 3 commits March 26, 2020 15:34

Apply suggestions from code review

3ae1f9d

Co-Authored-By: Adrin Jalali <adrin.jalali@gmail.com>

put back links + addressed rest of comments

4f4238b

Merge branch 'svm_UG' of github.com:NicolasHug/scikit-learn into svm_UG

4f0e34c

NicolasHug commented Mar 26, 2020

View reviewed changes

small detail about margin boundaries and SVs

035480b

NicolasHug added this to the 0.23 milestone Mar 31, 2020

thomasjpfan approved these changes Apr 6, 2020

View reviewed changes

doc/modules/svm.rst Outdated Show resolved Hide resolved

doc/modules/svm.rst Outdated Show resolved Hide resolved

adrinjalali reviewed Apr 6, 2020

View reviewed changes

NicolasHug added 2 commits April 6, 2020 07:58

Addressed comments from Thomas and Adrin

63f983c

Merge branch 'master' of github.com:scikit-learn/scikit-learn into sv…

d0b4211

…m_UG

adrinjalali approved these changes Apr 6, 2020

View reviewed changes

adrinjalali merged commit e3e9137 into scikit-learn:master Apr 6, 2020

		multi-class strategy, thus training n_class models. If there are only
		two classes, only one model is trained::

		:class:`LinearSVC` by the `liblinear`_ implementation is much more
		efficient than its `libsvm`_-based :class:`SVC` counterpart and can

	function for a separable problem, with three samples within the margin
	function for a linearly separable problem, with three samples on the margin

		While SVM models derived from `libsvm`_ and `liblinear`_ use ``C`` as
		While SVM models derived from `libsvm` and `liblinear` use ``C`` as

Uh oh!

Conversation

NicolasHug commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicolasHug commented Mar 26, 2020

Uh oh!

adrinjalali left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

thomasjpfan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

NicolasHug commented Mar 25, 2020 •

edited

Loading