Skip to content

[MPS] adding weight_norm_interface support for mps#108008

Closed
igm503 wants to merge 8 commits intopytorch:mainfrom
igm503:mpw_weight_norm
Closed

[MPS] adding weight_norm_interface support for mps#108008
igm503 wants to merge 8 commits intopytorch:mainfrom
igm503:mpw_weight_norm

Conversation

@igm503
Copy link
Contributor

@igm503 igm503 commented Aug 26, 2023 •
edited
Loading

Fixes #104513

Adds support for aten::_weight_norm_interface to the mps backend.

Also adds a consistency test for the output and the grad.

@pytorch-bot pytorch-bot bot added ciflow/mps Run MPS tests (subset of trunk) release notes: mps Release notes category labels Aug 26, 2023
@pytorch-bot
Copy link

pytorch-bot bot commented Aug 26, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/108008

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit bd79d8d with merge base 0bf30c1 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@igm503 igm503 closed this Sep 2, 2023
@igm503 igm503 deleted the mpw_weight_norm branch September 2, 2023 20:08
@igm503 igm503 restored the mpw_weight_norm branch September 2, 2023 20:13
@igm503 igm503 reopened this Sep 2, 2023
@igm503 igm503 marked this pull request as ready for review September 4, 2023 01:47
@igm503 igm503 requested a review from kulinseth as a code owner September 4, 2023 01:47
@igm503 igm503 changed the title adding weight_norm_interface support for mps [MPS] adding weight_norm_interface support for mps Sep 4, 2023
@soulitzer soulitzer added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Sep 4, 2023
@igm503
Copy link
Contributor Author

igm503 commented Sep 19, 2023

@kulinseth This should be good to go now. I've added a test in test/test_mps.py

kulinseth
kulinseth approved these changes Sep 19, 2023
View reviewed changes
Copy link
Collaborator

@kulinseth kulinseth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great thanks !!

@igm503
Copy link
Contributor Author

igm503 commented Sep 19, 2023

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 19, 2023
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Sep 19, 2023

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@varunneal varunneal mentioned this pull request Nov 9, 2023
Open
@fakerybakery
Copy link

fakerybakery commented Nov 16, 2023

Hi, for some reason this still isn't working...

@fakerybakery
Copy link

fakerybakery commented Nov 16, 2023

I'm on 2.1.1 and getting:

NotImplementedError: The operator 'aten::_weight_norm_interface' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

@igm503
Copy link
Contributor Author

igm503 commented Nov 16, 2023

General MPS op coverage tracking issue #77764

I'm on 2.1.1 and getting:

NotImplementedError: The operator 'aten::_weight_norm_interface' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

@fakerybakery This wasn't added in time for 2.1.1. It should be in the next release of PyTorch. In the meantime, you should be able to use it in the latest nightly versions of PyTorch.

@fakerybakery
Copy link

fakerybakery commented Nov 17, 2023

Ok, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kulinseth kulinseth kulinseth approved these changes

Assignees

No one assigned

Labels

ciflow/mps Run MPS tests (subset of trunk) ciflow/trunk Trigger trunk jobs on your pull request Merged open source release notes: mps Release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

implement weight_norm on mps

6 participants

@igm503 @pytorchmergebot @fakerybakery @kulinseth @soulitzer @pytorchbot