Fake Tensor Part 1 by eellison · Pull Request #77969 · pytorch/pytorch

eellison · 2022-05-20T15:42:14Z

Stack from ghstack (oldest at bottom):

This is just copying over the PR from albanD/subclass_zoo#32, and pulling in BaseTensor, plus adding one test based on invariants about schemas.

From code comments

Meta tensors give you the ability to run PyTorch code without having to
actually do computation through tensors allocated on a meta device.
Because the device is meta, meta tensors do not model device propagation.
FakeTensor extends MetaTensors to also carry an additional fake_device
which tracks devices that would have been used.

[ghstack-poisoned]

facebook-github-bot · 2022-05-20T15:42:21Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/77969
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours

✅ No Failures (0 Pending)

As of commit c8e78db (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

This is just copying over the PR from albanD/subclass_zoo#32, and pulling in `BaseTensor`, plus adding one test based on invariants about schemas. From code comments > Meta tensors give you the ability to run PyTorch code without having to actually do computation through tensors allocated on a `meta` device. Because the device is `meta`, meta tensors do not model device propagation. FakeTensor extends MetaTensors to also carry an additional `fake_device` which tracks devices that would have been used. [ghstack-poisoned]

ezyang · 2022-05-21T00:51:34Z

test/test_fake_tensor.py

@@ -0,0 +1,80 @@
+# Owner(s): ["module: unknown"]


well, could always chuck it in module: meta tensors

test_meta is under module: primTorch.. I dont think there is a meta module

there is haha module: meta tensors

torch/utils/_mode_utils.py

torch/_subclasses/__init__.py

ezyang · 2022-05-21T01:23:17Z

torch/_subclasses/base_tensor.py

+    # To ensure constructors can cooperate with one another, must accept and
+    # ignore element tensor (TODO: is this right???)
+    def __init__(self, elem):
+        super().__init__()


I'm pretty confident we can dump this constructor, it never actually got used for anything

ezyang · 2022-05-21T01:24:25Z

torch/_subclasses/base_tensor.py

+    # typically must be disabled
+    __torch_function__ = torch._C._disabled_torch_function_impl
+
+__all__ = ["BaseTensor"]


My preference is to not add this class, and inline the only thing you need (in this case, __torch_function__ = torch._C._disabled_torch_function_impl)

My understanding from talking to various folks was the

@staticmethod def __new__(cls, elem, *, requires_grad=None): if requires_grad is None: return super().__new__(cls, elem) # type: ignore else: return cls._make_subclass(cls, elem, requires_grad)

part was necessary.. I think I'll file issue for removal of this in future PR and leave for now

Huh, that's surprising.

ezyang · 2022-05-21T01:25:08Z

torch/_subclasses/fake_tensor.py

+    aten.is_pinned.default,
+    aten.to.device,
+    aten.to.prim_Device,
+    aten._pin_memory.default,


@anjali411 Would this be a good operator tag? (Or maybe we should be able to figure this out from schema?)

We can already infer this information from the FunctionSchema by iterating through the inputs

>>> torch.ops.aten.to.device._schema.arguments [const Tensor& self, Device device, int dtype, bool non_blocking, bool copy, int? memory_format] >>> torch.ops.aten.to.device._schema.arguments[1] Device device >>> torch.ops.aten.to.device._schema.arguments[1].kwarg_only False

Yeaaa I have a test for that. Maybe I just should just get rid of the list and move over the checking logic

yeah we should have one source of truth

ezyang · 2022-05-21T01:26:44Z

torch/_subclasses/fake_tensor.py

+        # elem does not need to be recorded, because FakeTensor *is a* elem
+        assert elem.device.type == "meta"
+        device = device if isinstance(device, torch.device) else torch.device(device)
+        assert device.type != "meta"


You know... it might be ok for the inner device to be meta lol

That seems like something has gone wrong (lost what the actual device is)... maybe if it proves to be needed in the future we can change

torch/_subclasses/fake_tensor.py

This is just copying over the PR from albanD/subclass_zoo#32, and pulling in `BaseTensor`, plus adding one test based on invariants about schemas. From code comments > Meta tensors give you the ability to run PyTorch code without having to actually do computation through tensors allocated on a `meta` device. Because the device is `meta`, meta tensors do not model device propagation. FakeTensor extends MetaTensors to also carry an additional `fake_device` which tracks devices that would have been used. [ghstack-poisoned]

eellison · 2022-05-24T02:24:00Z

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This is just copying over the PR from albanD/subclass_zoo#32, and pulling in `BaseTensor`, plus adding one test based on invariants about schemas. From code comments > Meta tensors give you the ability to run PyTorch code without having to actually do computation through tensors allocated on a `meta` device. Because the device is `meta`, meta tensors do not model device propagation. FakeTensor extends MetaTensors to also carry an additional `fake_device` which tracks devices that would have been used. Differential Revision: [D36618467](https://our.internmc.facebook.com/intern/diff/D36618467) [ghstack-poisoned]

eellison · 2022-05-24T16:25:35Z

@eellison has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This is just copying over the PR from albanD/subclass_zoo#32, and pulling in `BaseTensor`, plus adding one test based on invariants about schemas. From code comments > Meta tensors give you the ability to run PyTorch code without having to actually do computation through tensors allocated on a `meta` device. Because the device is `meta`, meta tensors do not model device propagation. FakeTensor extends MetaTensors to also carry an additional `fake_device` which tracks devices that would have been used. Differential Revision: [D36618467](https://our.internmc.facebook.com/intern/diff/D36618467) [ghstack-poisoned]

This is just copying over the PR from albanD/subclass_zoo#32, and pulling in `BaseTensor`, plus adding one test based on invariants about schemas. From code comments > Meta tensors give you the ability to run PyTorch code without having to actually do computation through tensors allocated on a `meta` device. Because the device is `meta`, meta tensors do not model device propagation. FakeTensor extends MetaTensors to also carry an additional `fake_device` which tracks devices that would have been used. [ghstack-poisoned]

eellison · 2022-05-31T16:17:21Z

@pytorchbot merge this please

github-actions · 2022-05-31T16:21:24Z

Hey @eellison.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

Summary: Pull Request resolved: #77969 Approved by: https://github.com/ezyang Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/678213ead2fd6676344e29e927246c658c1dab5c Reviewed By: seemethere Differential Revision: D36784747 Pulled By: seemethere fbshipit-source-id: 85a75483d4a0bf7247368cd1bca0576a31173c62

Fake Tensor Part 1

5617001

[ghstack-poisoned]

facebook-github-bot added the cla signed label May 20, 2022

facebook-github-bot added oncall: jit Add this issue/PR to JIT oncall triage queue module: fx labels May 20, 2022

This was referenced May 20, 2022

[temp] add slow path for is_contiguous #77683

Closed

Add slow path for device #77684

Closed

Change FakeTensor constructor to use _make_subclass #77970

Closed

add support for type_as/_to_copy #77971

Closed

Add FakeTensorMode #77972

Closed

eellison requested review from Chillee, ezyang and zou3519 and removed request for Chillee and zou3519 May 20, 2022 16:17

Elias Ellison added 3 commits May 20, 2022 11:23

ezyang reviewed May 21, 2022

View reviewed changes

torch/utils/_mode_utils.py Show resolved Hide resolved

ezyang reviewed May 21, 2022

View reviewed changes

torch/_subclasses/__init__.py Outdated Show resolved Hide resolved

ezyang reviewed May 21, 2022

View reviewed changes

torch/_subclasses/fake_tensor.py Show resolved Hide resolved

ezyang approved these changes May 21, 2022

View reviewed changes

eellison mentioned this pull request May 23, 2022

Add Caching of Conversion to Fake/Meta tensors in FakeTensorMode #78090

Closed

Elias Ellison added 2 commits May 23, 2022 13:28

Elias Ellison added 3 commits May 24, 2022 15:59

This was referenced May 25, 2022

add non-kwarg device #78270

Closed

Modernize FakeTensorMode, throw on non-fake inputs #78516

Closed

add _like constructors #78517

Closed

This was referenced May 31, 2022

Add CPU Fallback #78522

Closed

refactor op handling to use register pattern #78523

Closed

eellison mentioned this pull request May 31, 2022

[WIP] non-fake inputs #78524

Closed

pytorchmergebot added the Merged label May 31, 2022

pytorchmergebot closed this in 678213e May 31, 2022

facebook-github-bot deleted the gh/eellison/292/head branch June 4, 2022 14:17

Conversation

eellison commented May 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented May 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

✅ No Failures (0 Pending)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

eellison commented May 24, 2022

Uh oh!

eellison commented May 24, 2022

Uh oh!

eellison commented May 31, 2022

Uh oh!

github-actions bot commented May 31, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

eellison commented May 20, 2022 •

edited

Loading

facebook-github-bot commented May 20, 2022 •

edited

Loading