Making ops c10 full: out overloads with default arguments by smessmer · Pull Request #49012 · pytorch/pytorch

smessmer · 2020-12-08T11:27:35Z

Stack from ghstack:

Remove use_c10_dispatcher: full lines #49259 Remove use_c10_dispatcher: full lines
Remove generated_unboxing_wrappers and setManuallyBoxedKernel #49251 Remove generated_unboxing_wrappers and setManuallyBoxedKernel
Remove .impl_UNBOXED() and functionalities associated with it #49220 Remove .impl_UNBOXED() and functionalities associated with it
Remove codegen logic to support non-c10-full ops #49164 Remove codegen logic to support non-c10-full ops
Making ops c10-full: list of optional tensors #49138 [wip] Making ops c10-full: list of optional tensors
Making ops c10-full: ConstQuantizerPtr #49150 Making ops c10-full: ConstQuantizerPtr
Making ops c10-full: Storage arguments #49146 Making ops c10-full: Storage arguments
Making ops c10-full: optional lists #49088 Making ops c10-full: optional lists
Making ops c10 full: optional out arguments #49083 Making ops c10 full: optional out arguments
Making ops c10-full: Generator arguments #49013 Making ops c10-full: Generator arguments
Making ops c10 full: out overloads with default arguments #49012 Making ops c10 full: out overloads with default arguments
Making ops c10-full: Dimname arguments #49008 Making ops c10-full: Dimname arguments
Making ops c10-full: ops blocked by manual registrations #49007 Making ops c10-full: ops blocked by manual registrations
Making ops c10 full: ops returning multiple out arguments #49006 Making ops c10 full: ops returning multiple out arguments
Make out ops c10-full (with hacky-wrapper) #48912 Make out ops c10-full (with hacky-wrapper)

For some reason we apply default arguments to the functions in at::native too. So when an out overload had default arguments,
we couldn't move the out argument to the end because of those default arguments preceding it.
This PR fixes that and makes out overloads with default arguments c10-full

Differential Revision: D25394605

For some reason we apply default arguments to the functions in at::native too. So when an out overload had default arguments, we couldn't move the out argument to the end because of those default arguments preceding it. This PR fixes that and makes out overloads with default arguments c10-full Differential Revision: [D25394605](https://our.internmc.facebook.com/intern/diff/D25394605/) [ghstack-poisoned]

For some reason we apply default arguments to the functions in at::native too. So when an out overload had default arguments, we couldn't move the out argument to the end because of those default arguments preceding it. This PR fixes that and makes out overloads with default arguments c10-full Differential Revision: [D25394605](https://our.internmc.facebook.com/intern/diff/D25394605/) ghstack-source-id: 118080081 Pull Request resolved: #49012

dr-ci · 2020-12-08T12:25:33Z

💊 CI failures summary and remediations

As of commit b544e0d (more details on the Dr. CI page):

1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

pytorch_xla_linux_bionic_py3_6_clang9_build (1/1)

Step: "Build" (full log | diagnosis details | 🔁 rerun)

Dec 15 18:45:41 Failed to generate ATEN bindings: ['/var/lib/jenkins/workspace/xla/scripts/generate_code.sh']

Dec 15 18:45:40 AtenXlaType function missed override: std::tuple<Tensor&, Tensor&> min_out(Tensor& min, Tensor& min_indices, const Tensor& self, int64_t dim, bool keepdim); // min_out(Tensor,Tensor,Tensor,int64_t,bool)->std::tuple<Tensor,Tensor>
Dec 15 18:45:40 Traceback (most recent call last):
Dec 15 18:45:40   File "/var/lib/jenkins/workspace/xla/scripts/gen.py", line 1172, in <module>
Dec 15 18:45:40     generate(args)
Dec 15 18:45:40   File "/var/lib/jenkins/workspace/xla/scripts/gen.py", line 1142, in generate
Dec 15 18:45:40     assert check_overrides(overrides, overridden)
Dec 15 18:45:40 AssertionError
Dec 15 18:45:41 Building torch_xla version: 1.6
Dec 15 18:45:41 XLA Commit ID: fe89172b2bd1c9a1c104bd2cbe565f30e6c8e328
Dec 15 18:45:41 PyTorch Commit ID: 124695e35e275d550c6941be6511e2e9442cc6f4
Dec 15 18:45:41 Failed to generate ATEN bindings: ['/var/lib/jenkins/workspace/xla/scripts/generate_code.sh']
Dec 15 18:45:41 =================== sccache compilation log ===================
Dec 15 18:45:41 + cleanup
Dec 15 18:45:41 + retcode=1
Dec 15 18:45:41 + set +x
Dec 15 18:45:41 =========== If your build fails, please take a look at the log above for possible reasons ===========
Dec 15 18:45:41 Compile requests                    4562
Dec 15 18:45:41 Compile requests executed           4268
Dec 15 18:45:41 Cache hits                          2969
Dec 15 18:45:41 Cache hits (C/C++)                  2969
Dec 15 18:45:41 Cache misses                        1281

XLA failure

Job pytorch_xla_linux_bionic_py3_6_clang9_build is failing. Please create an issue with title prefixed by `[PT_BREAK]` in `pytorch/xla` and link to to this PR. If you have questions, please reach out to @ailzhang / @dlibenzi / @JackCaoG.

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 92 times.

For some reason we apply default arguments to the functions in at::native too. So when an out overload had default arguments, we couldn't move the out argument to the end because of those default arguments preceding it. This PR fixes that and makes out overloads with default arguments c10-full Differential Revision: [D25394605](https://our.internmc.facebook.com/intern/diff/D25394605/) [ghstack-poisoned]

ezyang · 2020-12-08T17:48:10Z

tools/codegen/gen.py

+                # TODO We should consider if we just want to remove
+                # default arguments from all at::native functions
+                # but that would be a larger change because we need
+                # to change a lot of call sites


In particular, direct native:: calls rely on the defaulting quite a bit.

For some reason we apply default arguments to the functions in at::native too. So when an out overload had default arguments, we couldn't move the out argument to the end because of those default arguments preceding it. This PR fixes that and makes out overloads with default arguments c10-full Differential Revision: [D25394605](https://our.internmc.facebook.com/intern/diff/D25394605/) [ghstack-poisoned]

bhosmer

Accepting to unblock, but if I'm right re the inline comment we should really change that before landing

tools/codegen/gen.py

For some reason we apply default arguments to the functions in at::native too. So when an out overload had default arguments, we couldn't move the out argument to the end because of those default arguments preceding it. This PR fixes that and makes out overloads with default arguments c10-full Differential Revision: [D25394605](https://our.internmc.facebook.com/intern/diff/D25394605/) [ghstack-poisoned]

facebook-github-bot · 2020-12-15T19:23:02Z

This pull request has been merged in a6274c1.

facebook-github-bot added the cla signed label Dec 8, 2020

This was referenced Dec 8, 2020

Gaps for making template-unboxing work for all operators #32366

Closed

Making ops c10-full: Generator arguments #49013

Closed

smessmer requested review from bhosmer and ezyang December 8, 2020 13:44

ezyang reviewed Dec 8, 2020

View reviewed changes

ezyang approved these changes Dec 8, 2020

View reviewed changes

bhosmer approved these changes Dec 9, 2020

View reviewed changes

tools/codegen/gen.py Outdated Show resolved Hide resolved

This was referenced Dec 9, 2020

remove hacky_wrapper from BackendSelect #49079

Closed

Making ops c10 full: optional out arguments #49083

Closed

smessmer mentioned this pull request Dec 9, 2020

Making ops c10-full: optional lists #49088

Closed

smessmer added 2 commits December 9, 2020 19:04

smessmer mentioned this pull request Dec 10, 2020

Making ops c10-full: list of optional tensors #49138

Closed

This was referenced Dec 10, 2020

Making ops c10-full: Storage arguments #49146

Closed

Making ops c10-full: ConstQuantizerPtr #49150

Closed

smessmer added 2 commits December 9, 2020 23:17

smessmer mentioned this pull request Dec 10, 2020

Remove codegen logic to support non-c10-full ops #49164

Closed

smessmer added 4 commits December 10, 2020 11:18

smessmer mentioned this pull request Dec 11, 2020

Remove .impl_UNBOXED() and functionalities associated with it #49220

Closed

smessmer added 4 commits December 10, 2020 23:51

This was referenced Dec 11, 2020

Remove generated_unboxing_wrappers and setManuallyBoxedKernel #49251

Closed

Remove use_c10_dispatcher: full lines #49259

Closed

smessmer added 2 commits December 12, 2020 02:23

facebook-github-bot closed this in a6274c1 Dec 15, 2020

facebook-github-bot added the Merged label Dec 15, 2020

facebook-github-bot deleted the gh/smessmer/276/head branch December 19, 2020 15:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Making ops c10 full: out overloads with default arguments#49012

Making ops c10 full: out overloads with default arguments#49012
smessmer wants to merge 21 commits intogh/smessmer/276/basefrom
gh/smessmer/276/head

smessmer commented Dec 8, 2020 •

edited

Loading

Uh oh!

dr-ci bot commented Dec 8, 2020 •

edited by facebook-github-bot

Loading

Uh oh!

ezyang Dec 8, 2020

Uh oh!

bhosmer left a comment

Uh oh!

Uh oh!

facebook-github-bot commented Dec 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

smessmer commented Dec 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dr-ci bot commented Dec 8, 2020 • edited by facebook-github-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💊 CI failures summary and remediations

🕵️ 1 new failure recognized by patterns

pytorch_xla_linux_bionic_py3_6_clang9_build (1/1)

XLA failure

Job pytorch_xla_linux_bionic_py3_6_clang9_build is failing. Please create an issue with title prefixed by [PT_BREAK] in pytorch/xla and link to to this PR. If you have questions, please reach out to @ailzhang / @dlibenzi / @JackCaoG.

Uh oh!

ezyang Dec 8, 2020

Choose a reason for hiding this comment

Uh oh!

bhosmer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

facebook-github-bot commented Dec 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

smessmer commented Dec 8, 2020 •

edited

Loading

dr-ci bot commented Dec 8, 2020 •

edited by facebook-github-bot

Loading

Job pytorch_xla_linux_bionic_py3_6_clang9_build is failing. Please create an issue with title prefixed by `[PT_BREAK]` in `pytorch/xla` and link to to this PR. If you have questions, please reach out to @ailzhang / @dlibenzi / @JackCaoG.