Skip to content

DOC Specify primal/dual formulation in LogisticRegression#26294

Merged
lorentzenchr merged 9 commits intoscikit-learn:mainfrom
mlondschien:logistic-regression-documentation
Jul 25, 2023
Merged

DOC Specify primal/dual formulation in LogisticRegression#26294
lorentzenchr merged 9 commits intoscikit-learn:mainfrom
mlondschien:logistic-regression-documentation

Conversation

@mlondschien
Copy link
Copy Markdown
Contributor

What does this implement/fix? Explain your changes.

It is ambiguous which of the two formulations for (Logistic) Ridge is dual / primal. As there is no duality gap, each formulation is the dual of the other. Someone coming from optimization would typically consider the constrained variant "primal", whereas someone from machine learning would typically consider the regularized variant "primal". This PR removes this ambiguity by explicitly referring to the variants as "constrained" and "regularized".

@mlondschien mlondschien changed the title Specify primal/dual formulation in LogisticRegression DOC Specify primal/dual formulation in LogisticRegression Apr 28, 2023
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jul 5, 2023

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: 7f01cd7. Link to the linter CI: here

@mlondschien
Copy link
Copy Markdown
Contributor Author

Pinging some recent contributors to LogisticRegression: @jeremiedbb, @lorentzenchr, @glemaitre. Could you have a quick look?

@lorentzenchr
Copy link
Copy Markdown
Member

It would help to point to a formula (eg in the user guide) for the regularized one such that it is crystal clear.

@mlondschien
Copy link
Copy Markdown
Contributor Author

Thanks, @lorentzenchr. I found a regularized formulation at https://scikit-learn.org/stable/modules/linear_model.html#binary-case. How would I link this and refer to an equation?

@lorentzenchr
Copy link
Copy Markdown
Member

lorentzenchr commented Jul 18, 2023

@glemaitre @lucyleeow Could you help out how to link to a formula in the user guide?

@lucyleeow
Copy link
Copy Markdown
Member

Maybe :math:numref: would work? (Would need to add a :label: to the math equation in linear_model.rst)

Otherwise, potentially :name: may work for a math block too, e.g. amending the equation to format,

.. math::
    :name: <eq name>

:math:numref: should be the way to go though!

@lucyleeow
Copy link
Copy Markdown
Member

Hmm it does look a bit odd now that this specific equation is the only one on the page that has a number next to it. Maybe :name: would be better?

.. math::
    :name: <name>

I need to refer to this :ref:`eq title <name>`

@mlondschien
Copy link
Copy Markdown
Contributor Author

mlondschien commented Jul 19, 2023

With all three approaches

.. math:: \min_{w} C \sum_{i=1}^n \left(-y_i \log(\hat{p}(X_i)) - (1 - y_i) \log(1 - \hat{p}(X_i))\right) + r(w).
   :label: logreg-regularized
.. math::
   :label: logreg-regularized
   
    \min_{w} C \sum_{i=1}^n \left(-y_i \log(\hat{p}(X_i)) - (1 - y_i) \log(1 - \hat{p}(X_i))\right) + r(w).

and

.. math::
    :name: regularized-logisticloss
   
    \min_{w} C \sum_{i=1}^n \left(-y_i \log(\hat{p}(X_i)) - (1 - y_i) \log(1 - \hat{p}(X_i))\right) + r(w).

the equation receives a number (1) in the top-left.

@lucyleeow
Copy link
Copy Markdown
Member

Thanks for testing it all! I would just go with :label: and accept the number.

We could think about working on adding labels to all equations in our docs WDYT @lorentzenchr ?

@lorentzenchr
Copy link
Copy Markdown
Member

@lucyleeow First of all, thanks for helping out.

I would not add labels to all equations we have, but make more use of labels on a case by case where needed base.

@lucyleeow
Copy link
Copy Markdown
Member

No worries, probably unnecessary. Just thought the numbering looked odd, especially as it isn't the first equation in the page. No matter.

@mlondschien
Copy link
Copy Markdown
Contributor Author

I've added the same text to LogisticRegressionCV.

@mlondschien
Copy link
Copy Markdown
Contributor Author

@lucyleeow @lorentzenchr Is there anything still needed to do before we can merge this?

@lucyleeow lucyleeow added the Quick Review For PRs that are quick to review label Jul 25, 2023
Copy link
Copy Markdown
Member

@lorentzenchr lorentzenchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
@mlondschien Thanks for this doc improvement.

@lorentzenchr lorentzenchr merged commit 507095b into scikit-learn:main Jul 25, 2023
@mlondschien mlondschien deleted the logistic-regression-documentation branch July 25, 2023 14:07
punndcoder28 pushed a commit to punndcoder28/scikit-learn that referenced this pull request Jul 29, 2023
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Sep 18, 2023
REDVM pushed a commit to REDVM/scikit-learn that referenced this pull request Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants