Add temporary impl_UNBOXED syntax sugar for unboxed-only defs. by ezyang · Pull Request #35714 · pytorch/pytorch

ezyang · 2020-03-30T22:03:24Z

Stack from ghstack:

Add temporary impl_UNBOXED syntax sugar for unboxed-only defs. #35714 Add temporary impl_UNBOXED syntax sugar for unboxed-only defs.
Add DispatchKey impl overload; remove use of torch::dispatch #35706 Add DispatchKey impl overload; remove use of torch::dispatch
Add unboxedCallRedispatch #35476 Add unboxedCallRedispatch

There are a lot of unboxed only defs. We're committed to removing
them at the end of the half but as I am about to do a lot of porting
to the new API, let's get them into a form where they're easy to
remove. This is a new overload impl_UNBOXED that will pass
the function pointer straight to CppFunction::makeUnboxedOnly

I don't attempt to make the _UNBOXED API complete; in particular,
catchall declarations don't get this sugar (as there are very few
of them).

Signed-off-by: Edward Z. Yang ezyang@fb.com

Differential Revision: D20775782

There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> [ghstack-poisoned]

There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> ghstack-source-id: a315082 Pull Request resolved: #35714

dr-ci · 2020-03-30T22:05:22Z

💊 CircleCI build failures summary and remediations

As of commit 633ac27 (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following build failures do not appear to be due to upstream breakages:

pytorch_linux_xenial_py3_6_gcc5_4_test (1/1)

Step: "Test" (full log | pattern match details)

Apr 01 17:59:04 [E request_callback_impl.cpp:94] Received error while processing request type 2: PickleError: ScriptModules cannot be deepcopied using copy.deepcopy or saved using torch.save. Mixed serialization of script and non-script modules is not supported. For purely script modules use my_script_module.save() instead.

Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(86): serialize 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(135): serialize 
Apr 01 17:59:04  
Apr 01 17:59:04 [E request_callback_impl.cpp:94] Received error while processing request type 2: PickleError: ScriptModules cannot be deepcopied using copy.deepcopy or saved using torch.save. Mixed serialization of script and non-script modules is not supported. For purely script modules use my_script_module.save(<filename>) instead. 
Apr 01 17:59:04  
Apr 01 17:59:04 At: 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/jit/__init__.py(1773): __getstate__ 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(86): serialize 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(135): serialize 
Apr 01 17:59:04  
Apr 01 17:59:04 [E request_callback_impl.cpp:94] Received error while processing request type 2: PickleError: ScriptModules cannot be deepcopied using copy.deepcopy or saved using torch.save. Mixed serialization of script and non-script modules is not supported. For purely script modules use my_script_module.save(<filename>) instead. 
Apr 01 17:59:04  
Apr 01 17:59:04 At: 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/jit/__init__.py(1773): __getstate__ 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(86): serialize 
Apr 01 17:59:04   /opt/conda/lib/python3.6/site-packages/torch/distributed/rpc/internal.py(135): serialize 
Apr 01 17:59:04  
Apr 01 17:59:04 ok (1.120s) 
Apr 01 17:59:05   test_unexepected_kwarg_is_specified (__main__.JitRpcTestWithSpawn) ... ok (1.118s) 
Apr 01 17:59:06   test_user_rrefs_confirmed (__main__.JitRpcTestWithSpawn) ... ok (1.119s) 
Apr 01 17:59:07   test_user_rrefs_confirmed_remote (__main__.JitRpcTestWithSpawn) ... ok (1.118s)

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 17 times.

smessmer · 2020-03-31T19:05:25Z

+  // CppFunction::makeUnboxedOnly factory directly (the API is compositional;
+  // it's just that we have a LOT of impl calls that are unboxed, so there is
+  // just a little syntax sugar for this case.)
+  template <typename Dispatch, typename Func>


Can we add a TODO here saying we intend to remove these?

…efs." There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> [ghstack-poisoned]

There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> ghstack-source-id: 1912dd1 Pull Request resolved: #35714

…efs." There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> Differential Revision: [D20775782](https://our.internmc.facebook.com/intern/diff/D20775782) [ghstack-poisoned]

There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> ghstack-source-id: c9e0507 Pull Request resolved: #35714

facebook-github-bot · 2020-04-02T19:03:23Z

@ezyang merged this pull request in 8e951c5.

…ed-only defs." Previously #35714 There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). To get some coverage of _UNBOXED API for code analysis, I switched one of our unboxed tests to be an impl rather than a def. This shouldn't materially affect coverage. Signed-off-by: Edward Z. Yang <ezyang@fb.com> [ghstack-poisoned]

…ed-only defs." Previously #35714 There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). To get some coverage of _UNBOXED API for code analysis, I switched one of our unboxed tests to be an impl rather than a def. This shouldn't materially affect coverage. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Differential Revision: [D20929259](https://our.internmc.facebook.com/intern/diff/D20929259) [ghstack-poisoned]

…s. (#36223) Summary: Pull Request resolved: #36223 Previously #35714 There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). To get some coverage of _UNBOXED API for code analysis, I switched one of our unboxed tests to be an impl rather than a def. This shouldn't materially affect coverage. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Test Plan: Imported from OSS Differential Revision: D20929259 Pulled By: ezyang fbshipit-source-id: 72d2061b6c8a6afbcd392b47f53ade18de2f9184

…s. (pytorch#36223) Summary: Pull Request resolved: pytorch#36223 Previously pytorch#35714 There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). To get some coverage of _UNBOXED API for code analysis, I switched one of our unboxed tests to be an impl rather than a def. This shouldn't materially affect coverage. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Test Plan: Imported from OSS Differential Revision: D20929259 Pulled By: ezyang fbshipit-source-id: 72d2061b6c8a6afbcd392b47f53ade18de2f9184

…ch#35714) Summary: Pull Request resolved: pytorch#35714 There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). Signed-off-by: Edward Z. Yang <ezyang@fb.com> Test Plan: Imported from OSS Differential Revision: D20775782 Pulled By: ezyang fbshipit-source-id: c5e804c69f5961c9d4862f6c5dbbe4c524cc32cc

…s. (pytorch#36223) Summary: Pull Request resolved: pytorch#36223 Previously pytorch#35714 There are a lot of unboxed only defs. We're committed to removing them at the end of the half but as I am about to do a lot of porting to the new API, let's get them into a form where they're easy to remove. This is a new overload impl_UNBOXED that will pass the function pointer straight to CppFunction::makeUnboxedOnly I don't attempt to make the _UNBOXED API complete; in particular, catchall declarations don't get this sugar (as there are very few of them). To get some coverage of _UNBOXED API for code analysis, I switched one of our unboxed tests to be an impl rather than a def. This shouldn't materially affect coverage. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Test Plan: Imported from OSS Differential Revision: D20929259 Pulled By: ezyang fbshipit-source-id: 72d2061b6c8a6afbcd392b47f53ade18de2f9184

This was referenced Mar 30, 2020

Disable c10::import(ns) #35398

Closed

Add unboxedCallRedispatch #35476

Closed

Add DispatchKey impl overload; remove use of torch::dispatch #35706

Closed

ezyang requested review from bhosmer and smessmer and removed request for smessmer March 30, 2020 22:06

bhosmer approved these changes Mar 30, 2020

View reviewed changes

smessmer approved these changes Mar 31, 2020

View reviewed changes

facebook-github-bot closed this in 8e951c5 Apr 2, 2020

facebook-github-bot added the merged label Apr 2, 2020

facebook-github-bot deleted the gh/ezyang/698/head branch April 6, 2020 14:17

ezyang mentioned this pull request Apr 8, 2020

[RELAND] Add temporary impl_UNBOXED syntax sugar for unboxed-only defs. #36223

Closed

mruberry added the Merged label Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add temporary impl_UNBOXED syntax sugar for unboxed-only defs.#35714

Add temporary impl_UNBOXED syntax sugar for unboxed-only defs.#35714
ezyang wants to merge 3 commits intogh/ezyang/698/basefrom
gh/ezyang/698/head

ezyang commented Mar 30, 2020 •

edited

Loading

Uh oh!

dr-ci Bot commented Mar 30, 2020 •

edited

Loading

Uh oh!

smessmer Mar 31, 2020

Uh oh!

ezyang Mar 31, 2020

Uh oh!

facebook-github-bot commented Apr 2, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

ezyang commented Mar 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci Bot commented Mar 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CircleCI build failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_linux_xenial_py3_6_gcc5_4_test (1/1)

Uh oh!

smessmer Mar 31, 2020

Choose a reason for hiding this comment

Uh oh!

ezyang Mar 31, 2020

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Apr 2, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ezyang commented Mar 30, 2020 •

edited

Loading

dr-ci Bot commented Mar 30, 2020 •

edited

Loading