Do not use TF32 matmul in linalg and DDP tests#56114
Closed
xwang233 wants to merge 10 commits intopytorch:masterfrom
Closed
Do not use TF32 matmul in linalg and DDP tests#56114xwang233 wants to merge 10 commits intopytorch:masterfrom
xwang233 wants to merge 10 commits intopytorch:masterfrom
Conversation
Contributor
💊 CI failures summary and remediationsAs of commit a83b61c (more details on the Dr. CI page):
1 failure not recognized by patterns:
🚧 1 fixed upstream failure:These were probably caused by upstream breakages that were already fixed.
Please rebase on the
|
Codecov Report
@@ Coverage Diff @@
## master #56114 +/- ##
==========================================
+ Coverage 76.44% 77.00% +0.56%
==========================================
Files 1990 1912 -78
Lines 199690 189561 -10129
==========================================
- Hits 152651 145980 -6671
+ Misses 47039 43581 -3458 |
Collaborator
Author
|
ping @ngimel 😄 |
ngimel
reviewed
May 19, 2021
| super(self.__class__, self).setUp() | ||
| torch.backends.cuda.matmul.allow_tf32 = False | ||
| self.precision_overrides = { | ||
| torch.float: 1e-4, |
Collaborator
There was a problem hiding this comment.
Does it mean that regular fp32 needs expanded tolerance? How are tests passing currently?
Collaborator
Author
There was a problem hiding this comment.
Ohh, yes you're correct. I forgot to delete them. Let me modify this.
ngimel
approved these changes
May 19, 2021
Contributor
|
@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator. |
Contributor
laurentdupin
pushed a commit
to laurentdupin/pytorch
that referenced
this pull request
Apr 25, 2026
Summary: This PR does several things to relax test tolerance - Do not use TF32 in cuda matmul in test_c10d. See pytorch#52941. - Do not use TF32 in cuda matmul in test_linalg. Increase atol for float and cfloat. See pytorch#50453 The tolerance is increased because most linear algebra operators are not that stable in single precision. Pull Request resolved: pytorch#56114 Reviewed By: ailzhang Differential Revision: D28554467 Pulled By: ngimel fbshipit-source-id: 90416be8e4c048bedb16903b01315584d344ecdf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR does several things to relax test tolerance
The tolerance is increased because most linear algebra operators are not that stable in single precision.