Add manual_cpp_binding to native_functions.yaml#49092
Closed
ezyang wants to merge 3 commits intogh/ezyang/889/basefrom
Closed
Add manual_cpp_binding to native_functions.yaml#49092ezyang wants to merge 3 commits intogh/ezyang/889/basefrom
ezyang wants to merge 3 commits intogh/ezyang/889/basefrom
Conversation
Functions which specify manual_cpp_binding don't automatically get C++ bindings generated for them in TensorBody.h or Functions.h. This lets end users manually define the bindings themselves, which may be helpful if there is a way to short circuit the dispatcher entirely. contiguous() is switched to use this mechanism. Although manual_cpp_binding suggests that we don't generate the binding at all, it is often the case that there is some "fast path", but when this path is not satisfied, we should go back to the slow dispatch. So we still generate a fallback method/function which the user-defined binding can call into in case that we have to go slowpath. The correctness conditions for bindings manually written in this way are subtle. Here are the ones I can think of off the top of my head: - Whatever condition is tested in the C++ body, must ALSO be tested again in the native:: implementation on the other side of the dispatcher. This is because you are NOT GUARANTEED to hit the native:: implementation through the C++ binding, you may go straight to the implementation via a boxed call. - If a binding is written in this way, it is only safe to skip dispatch if you would have returned the same tensor as before. In any situation you would return a fresh tensor, you MUST go to the slow path, because you need to actually get to the autograd kernel. Signed-off-by: Edward Z. Yang <ezyang@fb.com> [ghstack-poisoned]
ezyang
added a commit
that referenced
this pull request
Dec 9, 2020
Functions which specify manual_cpp_binding don't automatically get C++ bindings generated for them in TensorBody.h or Functions.h. This lets end users manually define the bindings themselves, which may be helpful if there is a way to short circuit the dispatcher entirely. contiguous() is switched to use this mechanism. Although manual_cpp_binding suggests that we don't generate the binding at all, it is often the case that there is some "fast path", but when this path is not satisfied, we should go back to the slow dispatch. So we still generate a fallback method/function which the user-defined binding can call into in case that we have to go slowpath. The correctness conditions for bindings manually written in this way are subtle. Here are the ones I can think of off the top of my head: - Whatever condition is tested in the C++ body, must ALSO be tested again in the native:: implementation on the other side of the dispatcher. This is because you are NOT GUARANTEED to hit the native:: implementation through the C++ binding, you may go straight to the implementation via a boxed call. - If a binding is written in this way, it is only safe to skip dispatch if you would have returned the same tensor as before. In any situation you would return a fresh tensor, you MUST go to the slow path, because you need to actually get to the autograd kernel. Signed-off-by: Edward Z. Yang <ezyang@fb.com> ghstack-source-id: c7a41a6 Pull Request resolved: #49092
Contributor
|
So how would ? |
bhosmer
approved these changes
Dec 9, 2020
|
|
||
| # Whether or not to skip generating TensorMethod/Functions bindings | ||
| # for this kernel. Technically, this doesn't actually skip generating | ||
| # the binding; instead, the binding gets generated to _slow_{funcname} |
Contributor
Author
For size, I'd go straight to TensorImpl method call, and bypass native:: entirely. Should all work out. |
Functions which specify manual_cpp_binding don't automatically get C++ bindings generated for them in TensorBody.h or Functions.h. This lets end users manually define the bindings themselves, which may be helpful if there is a way to short circuit the dispatcher entirely. contiguous() is switched to use this mechanism. Although manual_cpp_binding suggests that we don't generate the binding at all, it is often the case that there is some "fast path", but when this path is not satisfied, we should go back to the slow dispatch. So we still generate a fallback method/function which the user-defined binding can call into in case that we have to go slowpath. The correctness conditions for bindings manually written in this way are subtle. Here are the ones I can think of off the top of my head: - Whatever condition is tested in the C++ body, must ALSO be tested again in the native:: implementation on the other side of the dispatcher. This is because you are NOT GUARANTEED to hit the native:: implementation through the C++ binding, you may go straight to the implementation via a boxed call. - If a binding is written in this way, it is only safe to skip dispatch if you would have returned the same tensor as before. In any situation you would return a fresh tensor, you MUST go to the slow path, because you need to actually get to the autograd kernel. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Differential Revision: [D25428440](https://our.internmc.facebook.com/intern/diff/D25428440) [ghstack-poisoned]
Functions which specify manual_cpp_binding don't automatically get C++ bindings generated for them in TensorBody.h or Functions.h. This lets end users manually define the bindings themselves, which may be helpful if there is a way to short circuit the dispatcher entirely. contiguous() is switched to use this mechanism. Although manual_cpp_binding suggests that we don't generate the binding at all, it is often the case that there is some "fast path", but when this path is not satisfied, we should go back to the slow dispatch. So we still generate a fallback method/function which the user-defined binding can call into in case that we have to go slowpath. The correctness conditions for bindings manually written in this way are subtle. Here are the ones I can think of off the top of my head: - Whatever condition is tested in the C++ body, must ALSO be tested again in the native:: implementation on the other side of the dispatcher. This is because you are NOT GUARANTEED to hit the native:: implementation through the C++ binding, you may go straight to the implementation via a boxed call. - If a binding is written in this way, it is only safe to skip dispatch if you would have returned the same tensor as before. In any situation you would return a fresh tensor, you MUST go to the slow path, because you need to actually get to the autograd kernel. Signed-off-by: Edward Z. Yang <ezyang@fb.com> Differential Revision: [D25428440](https://our.internmc.facebook.com/intern/diff/D25428440) [ghstack-poisoned]
Contributor
1 similar comment
Contributor
This was referenced Dec 14, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Functions which specify manual_cpp_binding don't automatically
get C++ bindings generated for them in TensorBody.h or
Functions.h. This lets end users manually define the bindings
themselves, which may be helpful if there is a way to
short circuit the dispatcher entirely. contiguous() is switched
to use this mechanism.
Although manual_cpp_binding suggests that we don't generate the
binding at all, it is often the case that there is some "fast
path", but when this path is not satisfied, we should go back
to the slow dispatch. So we still generate a fallback method/function
which the user-defined binding can call into in case that we
have to go slowpath.
The correctness conditions for bindings manually written in this
way are subtle. Here are the ones I can think of off the top
of my head:
Whatever condition is tested in the C++ body, must ALSO be
tested again in the native:: implementation on the other
side of the dispatcher. This is because you are NOT GUARANTEED
to hit the native:: implementation through the C++ binding,
you may go straight to the implementation via a boxed call.
If a binding is written in this way, it is only safe to
skip dispatch if you would have returned the same tensor as
before. In any situation you would return a fresh tensor,
you MUST go to the slow path, because you need to actually
get to the autograd kernel.
Signed-off-by: Edward Z. Yang ezyang@fb.com
Differential Revision: D25428440