Disable TF32 in pinv_jvp and pinv_backward#67948
Closed
crcrpar wants to merge 1 commit intopytorch:masterfrom
Closed
Disable TF32 in pinv_jvp and pinv_backward#67948crcrpar wants to merge 1 commit intopytorch:masterfrom
pinv_jvp and pinv_backward#67948crcrpar wants to merge 1 commit intopytorch:masterfrom
Conversation
CI Flow Status⚛️ CI FlowRuleset - Version:
You can add a comment to the PR and tag @pytorchbot with the following commands: # ciflow rerun, "ciflow/default" will always be added automatically
@pytorchbot ciflow rerun
# ciflow rerun with additional labels "-l <ciflow/label_name>", which is equivalent to adding these labels manually and trigger the rerun
@pytorchbot ciflow rerun -l ciflow/scheduled -l ciflow/slowFor more information, please take a look at the CI Flow Wiki. |
Contributor
🔗 Helpful links
💊 CI failures summary and remediationsAs of commit 66f27a2 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
Collaborator
|
I'll merge this, but can we have a more systematic solution? A lot of linear algebra functions use matmuls in the backward. We currently disable tf32 in cublas handle that magma uses, but I think not for any other circumstances for linalg functions, and we don't run ampere in CI. |
ngimel
approved these changes
Nov 8, 2021
Contributor
|
@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Collaborator
|
Do you have more context on when it is ok to use tf32 and when not? (like a doc/slides) |
Collaborator
|
In linear algebra - never. The only context where it's ok to use tf32 is in linear layers in nn networks. |
Collaborator
|
Should we change the default then to never use it except when used by |
Collaborator
|
There;s #67384 discussing it, where using it only in nn.functional.linear is listed as option 3. However it's way to confusing when |
ngimel
approved these changes
Nov 8, 2021
Contributor
|
@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Contributor
facebook-github-bot
pushed a commit
that referenced
this pull request
Feb 28, 2022
cyyever
pushed a commit
to cyyever/pytorch_private
that referenced
this pull request
Mar 3, 2022
Summary: Disable TF32 in some linalg functions See also pytorch/pytorch#67948 #50453 pytorch/pytorch#44240 Pull Request resolved: pytorch/pytorch#73460 Reviewed By: albanD Differential Revision: D34493487 Pulled By: ngimel fbshipit-source-id: 958cd968ea09df3b5a4d2b4a26aaf0dfddc53981 (cherry picked from commit cd75ec645b86c4b4a66c35696ce891d006f3833b)
cyyever
pushed a commit
to cyyever/pytorch_private
that referenced
this pull request
Mar 3, 2022
Summary: Disable TF32 in some linalg functions See also pytorch/pytorch#67948 #50453 pytorch/pytorch#44240 Pull Request resolved: pytorch/pytorch#73460 Reviewed By: albanD Differential Revision: D34493487 Pulled By: ngimel fbshipit-source-id: 958cd968ea09df3b5a4d2b4a26aaf0dfddc53981 (cherry picked from commit cd75ec645b86c4b4a66c35696ce891d006f3833b)
laurentdupin
pushed a commit
to laurentdupin/pytorch
that referenced
this pull request
Apr 25, 2026
Summary: Fixes pytorch#67947 cc ptrblck xwang233 zasdfgbnm Pull Request resolved: pytorch#67948 Reviewed By: H-Huang Differential Revision: D32251934 Pulled By: ngimel fbshipit-source-id: a2b1a118337b38db61350c9e49f1ba19030d70ec
laurentdupin
pushed a commit
to laurentdupin/pytorch
that referenced
this pull request
Apr 25, 2026
Summary: Disable TF32 in some linalg functions See also pytorch#67948 pytorch#50453 pytorch#44240 Pull Request resolved: pytorch#73460 Reviewed By: albanD Differential Revision: D34493487 Pulled By: ngimel fbshipit-source-id: 958cd968ea09df3b5a4d2b4a26aaf0dfddc53981 (cherry picked from commit cd75ec6)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #67947
cc @ptrblck @xwang233 @zasdfgbnm