[UPDATED PROTOTYPE] Use dynamo fake tensor mode in aot_autograd, move aot_autograd compilation to lowering time by ezyang · Pull Request #89672 · pytorch/pytorch

ezyang · 2022-11-25T01:36:20Z

Stack from ghstack (oldest at bottom):

After all of the preparatory commits, this is a subset of the
changes in #89392 that actually
change us to propagating fake tensors to backends.

Signed-off-by: Edward Z. Yang ezyang@fb.com

cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire

… aot_autograd compilation to lowering time After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyang@fb.com> [ghstack-poisoned]

pytorch-bot · 2022-11-25T01:36:23Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/89672

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 Failures

As of commit f004e48:

The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…ograd, move aot_autograd compilation to lowering time" After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> cc mlazos soumith voznesenskym yanboliang penguinwu anijain2305 EikanWang jgong5 Guobing-Chen chunyuan-w XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire [ghstack-poisoned]

… aot_autograd compilation to lowering time After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> ghstack-source-id: 40f41ce Pull Request resolved: #89672

Chillee

Tentatively LGTM, might need to read more closely.

voznesenskym · 2022-11-28T17:44:23Z

Gonna look at tests on this one.

voznesenskym · 2022-11-28T17:44:37Z

Oh this is missing the C++ fix @ezyang

wconstab · 2022-11-28T18:37:20Z

torch/_dynamo/optimizations/distributed.py

+                            fake_submod = deepcopy_to_fake_tensor(real_mod, fake_mode)
+                        else:
+                            fake_submod = real_mod
+                            pass


when is not fake_mode? and why this pass?

bdhirsh · 2022-11-29T15:34:30Z

torch/_dynamo/output_graph.py

+            else:
+                # Fallback, in case fake_tensor was not set
+                # Particularly for graph args that are not tensors
+                result.extend(arg.get_examples())


Two questions about this comment:

(1) The return type of this func is List[torch.Tensor], is that wrong? (comment implies we have to worry about non-tensor graph args)

(2) Should this function maintain the invariant that it never returns real tensors, and assert here that get_examples() in the fallback only returns non-tensors?

…ograd, move aot_autograd compilation to lowering time" After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> cc mlazos soumith voznesenskym yanboliang penguinwu anijain2305 EikanWang jgong5 Guobing-Chen chunyuan-w XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire [ghstack-poisoned]

ezyang · 2022-12-01T04:47:52Z

torch/_dynamo/variables/builder.py

-                self.tx.output.graphargs.append(
-                    GraphArg(self.get_source(), value, False)
-                )
+            graph_arg = None


There's no need to define graph_arg here

…grad compilation to lowering time [Merger of 89672 and 89773]" After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> This is the merger of Ed's PR #89672, which is a rewrite of an older PR of mine (#89392), with CI Fixes on top of it (#89773) cc mlazos soumith yanboliang penguinwu anijain2305 EikanWang jgong5 Guobing-Chen chunyuan-w XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire [ghstack-poisoned]

…ation to lowering time [Merger of 89672 and 89773] (#90039) After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> This is the merger of Ed's PR #89672, which is a rewrite of an older PR of mine (#89392), with CI Fixes on top of it (#89773) Pull Request resolved: #90039 Approved by: https://github.com/ezyang

…ation to lowering time [Merger of 89672 and 89773] (#90039) After all of the preparatory commits, this is a subset of the changes in #89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> This is the merger of Ed's PR #89672, which is a rewrite of an older PR of mine (#89392), with CI Fixes on top of it (#89773) Pull Request resolved: #90039 Approved by: https://github.com/ezyang fix

…ation to lowering time [Merger of 89672 and 89773] (pytorch#90039) After all of the preparatory commits, this is a subset of the changes in pytorch#89392 that actually change us to propagating fake tensors to backends. Signed-off-by: Edward Z. Yang <ezyangfb.com> This is the merger of Ed's PR pytorch#89672, which is a rewrite of an older PR of mine (pytorch#89392), with CI Fixes on top of it (pytorch#89773) Pull Request resolved: pytorch#90039 Approved by: https://github.com/ezyang

ezyang · 2022-12-11T04:27:27Z

obsolete

ezyang requested review from H-Huang, awgu, kwen2501, mrshenli, pritamdamania87, rohan-varma and zhaojuanmao as code owners November 25, 2022 01:36

ezyang mentioned this pull request Nov 25, 2022

xfail maml test, instead of running it without fake tensor prop #89645

Closed

github-actions bot added ciflow/inductor module: dynamo module: inductor labels Nov 25, 2022

ezyang mentioned this pull request Nov 25, 2022

Use isinstance test rather than exact type test for wrap to fake #89671

Closed

github-actions bot requested review from Chillee, SherlockNoMad, albanD, anjali411, antoniojkim, bdhirsh, miladm and voznesenskym November 25, 2022 01:36

voznesenskym mentioned this pull request Nov 26, 2022

Unit test fixes #89703

Closed

voznesenskym mentioned this pull request Nov 27, 2022

Add simple assert to detect fake tensors on modules #89722

Closed

ezyang mentioned this pull request Nov 28, 2022

Refactor how AOTAutograd backends are defined #89736

Closed

Chillee reviewed Nov 28, 2022

View reviewed changes

voznesenskym mentioned this pull request Nov 28, 2022

Get CI passing #89773

Closed

wconstab reviewed Nov 28, 2022

View reviewed changes

bdhirsh reviewed Nov 29, 2022

View reviewed changes

albanD removed their request for review November 29, 2022 21:16

ezyang commented Dec 1, 2022

View reviewed changes

voznesenskym mentioned this pull request Dec 2, 2022

Use dynamo fake tensor mode in aot_autograd, move aot_autograd compilation to lowering time [Merger of 89672 and 89773] #90039

Closed

ezyang closed this Dec 11, 2022

facebook-github-bot deleted the gh/ezyang/1591/head branch June 8, 2023 16:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[UPDATED PROTOTYPE] Use dynamo fake tensor mode in aot_autograd, move aot_autograd compilation to lowering time#89672

[UPDATED PROTOTYPE] Use dynamo fake tensor mode in aot_autograd, move aot_autograd compilation to lowering time#89672
ezyang wants to merge 8 commits intogh/ezyang/1591/basefrom
gh/ezyang/1591/head

ezyang commented Nov 25, 2022 •

edited by voznesenskym

Loading

Uh oh!

pytorch-bot bot commented Nov 25, 2022 •

edited

Loading

Uh oh!

Chillee left a comment

Uh oh!

voznesenskym commented Nov 28, 2022

Uh oh!

voznesenskym commented Nov 28, 2022

Uh oh!

wconstab Nov 28, 2022

Uh oh!

bdhirsh Nov 29, 2022

Uh oh!

ezyang Dec 1, 2022

Uh oh!

ezyang commented Dec 11, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

ezyang commented Nov 25, 2022 • edited by voznesenskym Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Nov 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/89672

❌ 12 Failures

Uh oh!

Chillee left a comment

Choose a reason for hiding this comment

Uh oh!

voznesenskym commented Nov 28, 2022

Uh oh!

voznesenskym commented Nov 28, 2022

Uh oh!

wconstab Nov 28, 2022

Choose a reason for hiding this comment

Uh oh!

bdhirsh Nov 29, 2022

Choose a reason for hiding this comment

Uh oh!

ezyang Dec 1, 2022

Choose a reason for hiding this comment

Uh oh!

ezyang commented Dec 11, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ezyang commented Nov 25, 2022 •

edited by voznesenskym

Loading

pytorch-bot bot commented Nov 25, 2022 •

edited

Loading