[MPS] Enable bfloat16 testing by malfet · Pull Request #136987 · pytorch/pytorch

malfet · 2024-09-30T05:29:39Z

Stack from ghstack (oldest at bottom):

By even further reducing precisions of imprecise FP16 ops, introducing new BF16_LOW_PRECISION_OPS category and marking BF16 tests as xfail for divfloor_rounding, floor_divide and remainder.
I guess the nature of low-precision results, is that MPSGraph, unlike the rest of the PyTorch does not do accumulation over fp32 for reduction operations

[ghstack-poisoned]

pytorch-bot · 2024-09-30T05:29:43Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136987

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d3b8428 with merge base dfe1d45 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]

Before this change, test failed with unable to compile errors, as `bfloat16` requires explicit cast. Tested in #136987 Pull Request resolved: #136981 Approved by: https://github.com/Skylion007

) Just adds instantiation of the kernels and sometimes explicit cast. Tested in #136987 Pull Request resolved: #136982 Approved by: https://github.com/Skylion007 ghstack dependencies: #136981

By simply adding explicit instantiation Tested in #136987 Pull Request resolved: #136983 Approved by: https://github.com/Skylion007 ghstack dependencies: #136981, #136982

By adding explicit instantiation. Tested in #136987 Pull Request resolved: #136984 Approved by: https://github.com/Skylion007 ghstack dependencies: #136981, #136982, #136983

[ghstack-poisoned]

ghstack-source-id: 94bb429 Pull Request resolved: #136987

[ghstack-poisoned]

ghstack-source-id: abf4979 Pull Request resolved: #136987

By even further reducing precisions of imprecise FP16 ops, introducing new BF16_LOW_PRECISION_OPS category and marking BF16 tests as xfail for `divfloor_rounding`, `floor_divide` and `remainder` [ghstack-poisoned]

For Metal cast ops to comple, one need to explicitly cast to/from `bfloat` unlike for other dtypes Tested in #136987 [ghstack-poisoned]

albanD

Any chance we can fix that behavior? This will most likely change the numerics for end users significantly.
@kulinseth ?

For Metal cast ops to comple, one need to explicitly cast to/from `bfloat` unlike for other dtypes Tested in #136987 Pull Request resolved: #137070 Approved by: https://github.com/Skylion007

By even further reducing precisions of imprecise FP16 ops, introducing new BF16_LOW_PRECISION_OPS category and marking BF16 tests as xfail for `divfloor_rounding`, `floor_divide` and `remainder`. I guess the nature of low-precision results, is that MPSGraph, unlike the rest of the PyTorch does not do accumulation over fp32 for reduction operations [ghstack-poisoned]

ghstack-source-id: b999969 Pull Request resolved: #136987

malfet · 2024-10-01T17:08:27Z

@pytorchbot merge -f "MPS + Lint are green"

pytorchmergebot · 2024-10-01T17:09:59Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Before this change, test failed with unable to compile errors, as `bfloat16` requires explicit cast. Tested in pytorch#136987 Pull Request resolved: pytorch#136981 Approved by: https://github.com/Skylion007

…rch#136982) Just adds instantiation of the kernels and sometimes explicit cast. Tested in pytorch#136987 Pull Request resolved: pytorch#136982 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#136981

By simply adding explicit instantiation Tested in pytorch#136987 Pull Request resolved: pytorch#136983 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#136981, pytorch#136982

By adding explicit instantiation. Tested in pytorch#136987 Pull Request resolved: pytorch#136984 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#136981, pytorch#136982, pytorch#136983

For Metal cast ops to comple, one need to explicitly cast to/from `bfloat` unlike for other dtypes Tested in pytorch#136987 Pull Request resolved: pytorch#137070 Approved by: https://github.com/Skylion007

By even further reducing precisions of imprecise FP16 ops, introducing new BF16_LOW_PRECISION_OPS category and marking BF16 tests as xfail for `divfloor_rounding`, `floor_divide` and `remainder`. I guess the nature of low-precision results, is that MPSGraph, unlike the rest of the PyTorch does not do accumulation over fp32 for reduction operations Pull Request resolved: pytorch#136987 Approved by: https://github.com/albanD ghstack dependencies: pytorch#137070

[MPS] Enable bfloat16 testing

0ef8020

[ghstack-poisoned]

malfet requested a review from kulinseth as a code owner September 30, 2024 05:29

This was referenced Sep 30, 2024

[MPS] Fix Gamma for bfloat16 dtypes #136981

Closed

[MPS] Extend fmin/fmax/copysign and nextafter to blfoat #136982

Closed

malfet mentioned this pull request Sep 30, 2024

[MPS] Enable bucketization for bfloat16 #136983

Closed

pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) topic: not user facing topic category labels Sep 30, 2024

This was referenced Sep 30, 2024

[MPS] Enable torch.linalg.cross for bfloat16 #136984

Closed

[MPS] Enable Renorm for bfloat16 #136985

Closed

[MPS] Enable nan_to_num for bfloat16 #136986

Closed

malfet requested a review from Skylion007 September 30, 2024 05:30

malfet added topic: improvements topic category release notes: mps Release notes category topic: not user facing topic category and removed topic: not user facing topic category topic: improvements topic category release notes: mps Release notes category labels Sep 30, 2024

Update on "[MPS] Enable bfloat16 testing"

ce30645

[ghstack-poisoned]

This was referenced Sep 30, 2024

Testing: Unblock new_* testing on MPS #137003

Closed

[MPS][BE] Use None as alias for all types #137004

Closed

Update on "[MPS] Enable bfloat16 testing"

1709ba6

[ghstack-poisoned]

malfet added a commit that referenced this pull request Sep 30, 2024

[MPS] Enable bfloat16 testing

2441eef

ghstack-source-id: 94bb429 Pull Request resolved: #136987

Update on "[MPS] Enable bfloat16 testing"

7d8713a

[ghstack-poisoned]

malfet mentioned this pull request Oct 1, 2024

[MPS] Fix bfloat to complex casts #137070

Closed

malfet added a commit that referenced this pull request Oct 1, 2024

[MPS] Enable bfloat16 testing

0c9e422

ghstack-source-id: abf4979 Pull Request resolved: #136987

malfet requested a review from albanD October 1, 2024 01:55

Update on "[MPS] Enable bfloat16 testing"

8e41994

By even further reducing precisions of imprecise FP16 ops, introducing new BF16_LOW_PRECISION_OPS category and marking BF16 tests as xfail for `divfloor_rounding`, `floor_divide` and `remainder` [ghstack-poisoned]

malfet added a commit that referenced this pull request Oct 1, 2024

Update on "[MPS] Fix bfloat to complex casts"

ebf87c7

For Metal cast ops to comple, one need to explicitly cast to/from `bfloat` unlike for other dtypes Tested in #136987 [ghstack-poisoned]

albanD approved these changes Oct 1, 2024

View reviewed changes

malfet added a commit that referenced this pull request Oct 1, 2024

[MPS] Enable bfloat16 testing

00f4890

ghstack-source-id: b999969 Pull Request resolved: #136987

pytorchmergebot added the merging label Oct 1, 2024

pytorchmergebot added the Merged label Oct 1, 2024

pytorchmergebot closed this in 3d0cb81 Oct 1, 2024

pytorchmergebot removed the merging label Oct 1, 2024

github-actions bot deleted the gh/malfet/26/head branch November 6, 2024 02:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MPS] Enable bfloat16 testing#136987

[MPS] Enable bfloat16 testing#136987
malfet wants to merge 6 commits intogh/malfet/26/basefrom
gh/malfet/26/head

malfet commented Sep 30, 2024 •

edited

Loading

Uh oh!

pytorch-bot bot commented Sep 30, 2024 •

edited

Loading

Uh oh!

albanD left a comment

Uh oh!

malfet commented Oct 1, 2024

Uh oh!

pytorchmergebot commented Oct 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

malfet commented Sep 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136987

✅ No Failures

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

malfet commented Oct 1, 2024

Uh oh!

pytorchmergebot commented Oct 1, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

malfet commented Sep 30, 2024 •

edited

Loading

pytorch-bot bot commented Sep 30, 2024 •

edited

Loading