Nvfuser guard patch by jjsjann123 · Pull Request #75016 · pytorch/pytorch

jjsjann123 · 2022-03-31T13:35:37Z

Fixes issue where CudaFusionGuard would return false on backward graph because requires_grad flag doesn't match.

This is due to the fact that autodiff uses GradMode switch to turn on/off requires_grad, which is not taken into consideration by nvfuser guard. We verified the implementation under TensorType::matchTensor.

Add python test to verify no fallback is observed

facebook-github-bot · 2022-03-31T13:35:42Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/75016
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours

💊 CI failures summary and remediations

As of commit 1b0b917 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

eellison

LGTM

eellison · 2022-03-31T15:01:25Z

       (guard_tensor_type->device().value() != tensor.device())) ||
      (guard_tensor_type->requiresGrad().has_value() &&
-       guard_tensor_type->requiresGrad().value() != tensor.requires_grad())) {
+       guard_tensor_type->requiresGrad().value() !=


I made this mistake as well :/ aa99df5

Thanks for making me feel better~

eellison · 2022-03-31T17:31:15Z

have some failing tests

facebook-github-bot · 2022-03-31T21:09:31Z

@eellison has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

eellison · 2022-04-01T14:22:20Z

@pytorchbot merge this please

github-actions · 2022-04-01T14:24:34Z

Hey @jjsjann123.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

fixes merge issue between #75016 and #73322 [ghstack-poisoned]

fixes merge issue between #75016 and #73322 ghstack-source-id: a79f74a Pull Request resolved: #75134

malfet · 2022-04-01T23:44:39Z

@pytorchbot revert this

malfet · 2022-04-01T23:45:52Z

Temporary reverting in order to resolve land race between this PR and 96c8799
cc: @atalman , @eellison

This reverts commit d86181f. Reverted #75016 on behalf of https://github.com/malfet

davidberard98 · 2022-04-01T23:50:05Z

sorry didn't notice this when I landed the ci stuff, @jjsjann123 could you rebase this and also change RUN_CUDA -> RUN_NVFUSER to avoid rocm failures?

malfet · 2022-04-01T23:51:46Z

Oh, so it needed to be reverted anyway ;)

davidberard98 · 2022-04-01T23:57:49Z

yeah, either this or the CI changes would have needed to be reverted

Chillee · 2022-04-06T01:25:22Z

Will reland in #75303 if @jjsjann123 doesn't mind :)

jjsjann123 · 2022-04-06T01:50:07Z

Will reland in #75303 if @jjsjann123 doesn't mind :)

ohohoh, somehow haven't got back to this one. Thanks for taking care of it.

Chillee · 2022-04-06T02:02:44Z

np - useful for me :)

Reland of #75016 with `USE_CUDA` => `USE_NVFUSER` Pull Request resolved: #75303 Approved by: https://github.com/jjsjann123, https://github.com/davidberard98

Summary: Reland of #75016 with `USE_CUDA` => `USE_NVFUSER` Pull Request resolved: #75303 Approved by: https://github.com/jjsjann123, https://github.com/davidberard98 Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/5994d684840a6b3be37bfa033af92c891ba257a6 Reviewed By: b0noI, davidberard98 Differential Revision: D35420569 Pulled By: Chillee fbshipit-source-id: e25a26b85f5056ba7b5b73448b03aa9926ce00df

Fixes issue where CudaFusionGuard would return false on backward graph because `requires_grad` flag doesn't match. This is due to the fact that autodiff uses GradMode switch to turn on/off requires_grad, which is not taken into consideration by nvfuser guard. We verified the implementation under `TensorType::matchTensor`. - [x] Add python test to verify no fallback is observed Pull Request resolved: pytorch/pytorch#75016 Approved by: https://github.com/eellison

This reverts commit 685e552. Reverted pytorch/pytorch#75016 on behalf of https://github.com/malfet

Reland of pytorch/pytorch#75016 with `USE_CUDA` => `USE_NVFUSER` Pull Request resolved: pytorch/pytorch#75303 Approved by: https://github.com/jjsjann123, https://github.com/davidberard98

Fixes issue where CudaFusionGuard would return false on backward graph because `requires_grad` flag doesn't match. This is due to the fact that autodiff uses GradMode switch to turn on/off requires_grad, which is not taken into consideration by nvfuser guard. We verified the implementation under `TensorType::matchTensor`. - [x] Add python test to verify no fallback is observed Pull Request resolved: pytorch/pytorch#75016 Approved by: https://github.com/eellison

This reverts commit bc31a2b. Reverted pytorch/pytorch#75016 on behalf of https://github.com/malfet

Reland of pytorch/pytorch#75016 with `USE_CUDA` => `USE_NVFUSER` Pull Request resolved: pytorch/pytorch#75303 Approved by: https://github.com/jjsjann123, https://github.com/davidberard98

Fixes issue where CudaFusionGuard would return false on backward graph because `requires_grad` flag doesn't match. This is due to the fact that autodiff uses GradMode switch to turn on/off requires_grad, which is not taken into consideration by nvfuser guard. We verified the implementation under `TensorType::matchTensor`. - [x] Add python test to verify no fallback is observed Pull Request resolved: pytorch#75016 Approved by: https://github.com/eellison

This reverts commit d0ce1d1. Reverted pytorch#75016 on behalf of https://github.com/malfet

Reland of pytorch#75016 with `USE_CUDA` => `USE_NVFUSER` Pull Request resolved: pytorch#75303 Approved by: https://github.com/jjsjann123, https://github.com/davidberard98

jjsjann123 added 2 commits March 31, 2022 06:29

patching guard logic on requires_grad for autodiff gradMode switch

a9ba615

clang-format

05ebb75

jjsjann123 requested a review from Chillee March 31, 2022 13:35

facebook-github-bot added the cla signed label Mar 31, 2022

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Mar 31, 2022

pytorchbot added the open source label Mar 31, 2022

python tests added

e3e6a15

jjsjann123 requested a review from Krovatkin March 31, 2022 14:31

eellison approved these changes Mar 31, 2022

View reviewed changes

jjsjann123 added 2 commits March 31, 2022 13:46

flake8

35b5a23

Merge remote-tracking branch 'upstream/master' into HEAD

1b0b917

pytorchmergebot closed this in d86181f Apr 1, 2022

jjsjann123 deleted the nvfuser_guard_patch branch April 1, 2022 19:35

davidberard98 mentioned this pull request Apr 1, 2022

[JIT] RUN_CUDA -> RUN_NVFUSER to fix nvfuser ci #75134

Closed

davidberard98 added a commit that referenced this pull request Apr 1, 2022

[JIT] RUN_CUDA -> RUN_NVFUSER to fix nvfuser ci

379aac2

fixes merge issue between #75016 and #73322 [ghstack-poisoned]

davidberard98 added a commit that referenced this pull request Apr 1, 2022

Update on "[JIT] RUN_CUDA -> RUN_NVFUSER to fix nvfuser ci"

456aaa4

fixes merge issue between #75016 and #73322 [ghstack-poisoned]

davidberard98 added a commit that referenced this pull request Apr 1, 2022

[JIT] RUN_CUDA -> RUN_NVFUSER to fix nvfuser ci

857f21c

fixes merge issue between #75016 and #73322 ghstack-source-id: a79f74a Pull Request resolved: #75134

pytorchmergebot added the Reverted label Apr 1, 2022

pytorchmergebot added a commit that referenced this pull request Apr 1, 2022

Revert "Nvfuser guard patch"

1352c64

This reverts commit d86181f. Reverted #75016 on behalf of https://github.com/malfet

zengk95 mentioned this pull request Apr 5, 2022

[Meta] CI Revert Tracker #66178

Closed

Chillee mentioned this pull request Apr 6, 2022

Reland NVFuser guard changes #75303

Closed

pytorchmergebot pushed a commit that referenced this pull request Apr 6, 2022

Reland NVFuser guard changes

5994d68

Reland of #75016 with `USE_CUDA` => `USE_NVFUSER` Pull Request resolved: #75303 Approved by: https://github.com/jjsjann123, https://github.com/davidberard98

jjsjann123 pushed a commit to jjsjann123/nvfuser that referenced this pull request Oct 29, 2022

Revert "Nvfuser guard patch"

b3c447d

This reverts commit 685e552. Reverted pytorch/pytorch#75016 on behalf of https://github.com/malfet

jjsjann123 pushed a commit to jjsjann123/nvfuser that referenced this pull request Nov 10, 2022

Revert "Nvfuser guard patch"

eb9e2cb

This reverts commit bc31a2b. Reverted pytorch/pytorch#75016 on behalf of https://github.com/malfet

laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026

Revert "Nvfuser guard patch"

3f34ea7

This reverts commit d0ce1d1. Reverted pytorch#75016 on behalf of https://github.com/malfet

Conversation

jjsjann123 commented Mar 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Mar 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

💊 CI failures summary and remediations

Uh oh!

eellison left a comment

Choose a reason for hiding this comment

Uh oh!

eellison Mar 31, 2022

Choose a reason for hiding this comment

Uh oh!

jjsjann123 Mar 31, 2022

Choose a reason for hiding this comment

Uh oh!

eellison commented Mar 31, 2022

Uh oh!

facebook-github-bot commented Mar 31, 2022

Uh oh!

eellison commented Apr 1, 2022

Uh oh!

github-actions Bot commented Apr 1, 2022

Uh oh!

malfet commented Apr 1, 2022

Uh oh!

malfet commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidberard98 commented Apr 1, 2022

Uh oh!

malfet commented Apr 1, 2022

Uh oh!

davidberard98 commented Apr 1, 2022

Uh oh!

Chillee commented Apr 6, 2022

Uh oh!

jjsjann123 commented Apr 6, 2022

Uh oh!

Chillee commented Apr 6, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

jjsjann123 commented Mar 31, 2022 •

edited

Loading

facebook-github-bot commented Mar 31, 2022 •

edited

Loading

malfet commented Apr 1, 2022 •

edited

Loading