Better reshape with autograd support (#82754)#84154

Closed

drisspg wants to merge 1 commit intopytorch:masterfrom

drisspg:export-D39023822

Contributor

drisspg commented Aug 26, 2022 •

edited

Loading

The original author is @YifanShenSZ and the original PR is: #82754

Summary:

Previous reshape https://github.com/pytorch/pytorch/issues/80981 is ok for forward, but needs improvement for backward: need to handle "sometimes view sometimes copy" behavior.

This pull request fixes it by:

add a new alias dispatch key CompositeImplicitAutogradNestedTensor, which ideally would work as nested-tensor version of CompositeImplicitAutograd
register reshape_nested to reshape by CompositeImplicitAutogradNestedTensor

Side changes:

add contiguous memory format support to clone_nested
add view_nested
add reshape_as_nested

Fix issue #83041

Pull Request resolved: #82754

Test Plan:
Imported from GitHub, without a Test Plan: line.

Static Docs Preview: executorch
|Full Site|

|Modified Pages|

Reviewed By: albanD

Differential Revision: D39023822

Pulled By: drisspg

drisspg requested review from albanD and soulitzer as code owners

August 26, 2022 22:34

facebook-github-bot added the cla signed label

Contributor

facebook-github-bot commented Aug 26, 2022 •

edited

Loading

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/84154
✖️ Python docs build was skipped
✖️ C++ docs build was skipped
❓Need help or want to give feedback on the CI? Visit our office hours

✅ No Failures (0 Pending)

As of commit 2b2b75e (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

facebook-github-bot added the fb-exported label

Contributor

facebook-github-bot commented Aug 26, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg requested a review from bdhirsh

August 26, 2022 22:37

drisspg mentioned this pull request

[Nested Tensor]Nested reshape #83880

Closed

drisspg added module: nestedtensor release notes: nested tensor labels

Contributor

facebook-github-bot commented Aug 26, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg force-pushed the export-D39023822 branch from 3df38ee to eca1360 Compare

August 26, 2022 23:50

drisspg force-pushed the export-D39023822 branch from eca1360 to cf96a2f Compare

August 26, 2022 23:58

Contributor

facebook-github-bot commented Aug 26, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

1 similar comment

Contributor

facebook-github-bot commented Aug 27, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg force-pushed the export-D39023822 branch from cf96a2f to 13e574d Compare

August 27, 2022 00:57

Contributor

facebook-github-bot commented Aug 27, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg force-pushed the export-D39023822 branch from 13e574d to a746163 Compare

August 27, 2022 01:56

bdhirsh approved these changes

View reviewed changes

Collaborator

bdhirsh left a comment

lgtm!

albanD approved these changes

View reviewed changes

Collaborator

albanD left a comment

I want to look into the view logic in more details. But this should be ok to land for now.

Contributor

facebook-github-bot commented Aug 29, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg force-pushed the export-D39023822 branch from a746163 to e2d7f7f Compare

August 29, 2022 18:32

Contributor

facebook-github-bot commented Aug 29, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg force-pushed the export-D39023822 branch from e2d7f7f to c98f7da Compare

August 29, 2022 18:39


          Better reshape with autograd support (pytorch#84154)

2b2b75e

Summary:
Pull Request resolved: pytorch#84154

Previous reshape [https://github.com/pytorch/pytorch/issues/80981](https://github.com/pytorch/pytorch/pull/80981) is ok for forward, but needs improvement for backward: need to handle "sometimes view sometimes copy" behavior.

This pull request fixes it by:
1. add a new alias dispatch key `CompositeImplicitAutogradNestedTensor`, which ideally would work as nested-tensor version of `CompositeImplicitAutograd`
2. register `reshape_nested` to `reshape` by `CompositeImplicitAutogradNestedTensor`

Side changes:
* add contiguous memory format support to `clone_nested`
* add `view_nested`
* add `reshape_as_nested`

Fix issue [https://github.com/pytorch/pytorch/issues/83041](https://github.com/pytorch/pytorch/issues/83041)

Pull Request resolved: pytorch#82754

Test Plan:
Imported from GitHub, without a `Test Plan:` line.

**Static Docs Preview: executorch**
|[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D39023822/V26/executorch/)|

|**Modified Pages**|

**Static Docs Preview: executorch**
|[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D39023822/V17/executorch/)|

|**Modified Pages**|

Reviewed By: albanD, bdhirsh

Differential Revision: D39023822

Pulled By: drisspg

fbshipit-source-id: 87acd4f9fb61cd094fccad7801c25e2a1bfed88b

Contributor

facebook-github-bot commented Aug 31, 2022

This pull request was exported from Phabricator. Differential Revision: D39023822

drisspg force-pushed the export-D39023822 branch from c98f7da to 2b2b75e Compare

August 31, 2022 17:07

Contributor

facebook-github-bot commented Sep 1, 2022

@pytorchbot merge -f 'Landed internally'

(Initiating merge automatically since Phabricator Diff has merged, using force because this PR might not pass merge_rules.json but landed internally)

Collaborator

pytorchmergebot commented Sep 1, 2022

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the force (-f) flag. This means your change will be merged immediately, bypassing any CI checks (ETA: 1-5 minutes). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

pytorchmergebot added the Merged label

pytorchmergebot closed this in

673b35c

facebook-github-bot pushed a commit that referenced this pull request


          Better reshape with autograd support (#84154)

84e1e48

Summary:
Pull Request resolved: #84154

Previous reshape [https://github.com/pytorch/pytorch/issues/80981](https://github.com/pytorch/pytorch/pull/80981) is ok for forward, but needs improvement for backward: need to handle "sometimes view sometimes copy" behavior.

This pull request fixes it by:
1. add a new alias dispatch key `CompositeImplicitAutogradNestedTensor`, which ideally would work as nested-tensor version of `CompositeImplicitAutograd`
2. register `reshape_nested` to `reshape` by `CompositeImplicitAutogradNestedTensor`

Side changes:
* add contiguous memory format support to `clone_nested`
* add `view_nested`
* add `reshape_as_nested`

Fix issue [https://github.com/pytorch/pytorch/issues/83041](https://github.com/pytorch/pytorch/issues/83041)

Pull Request resolved: #82754

Test Plan:
Imported from GitHub, without a `Test Plan:` line.

**Static Docs Preview: executorch**
|[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D39023822/V27/executorch/)|

|**Modified Pages**|

**Static Docs Preview: executorch**
|[Full Site](https://our.intern.facebook.com/intern/staticdocs/eph/D39023822/V17/executorch/)|

|**Modified Pages**|

Reviewed By: albanD, bdhirsh

Differential Revision: D39023822

Pulled By: drisspg

fbshipit-source-id: 872c81dc847d280366ef9f187f9b9bcb06aac73f

YifanShenSZ mentioned this pull request

Codegen for AutogradNestedTensor #83041

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed fb-exported Merged module: nestedtensor release notes: nested tensor