[AOTInductor] Call most runtime fallback ops without calling into Python#154142
[AOTInductor] Call most runtime fallback ops without calling into Python#154142benjaminglass1 wants to merge 20 commits intogh/benjaminglass1/84/basefrom
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/154142
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 06bfb9c with merge base 2625c70 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Uses the new aoti_torch_call_dispatcher interface to call runtime fallback ops without calling back into Python. This supports a limited subset of input and output datatypes, but a large number of fallback ATen ops are covered. Fixes #150988 Fixes #153478 ghstack-source-id: 70cbe2a Pull Request resolved: #154142
Uses the new aoti_torch_call_dispatcher interface to call runtime fallback ops without calling back into Python. This supports a limited subset of input and output datatypes, but a large number of fallback ATen ops are covered. Fixes #150988 Fixes #153478 ghstack-source-id: dd3d05a Pull Request resolved: #154142
Uses the new aoti_torch_call_dispatcher interface to call runtime fallback ops without calling back into Python. This supports a limited subset of input and output datatypes, but a large number of fallback ATen ops are covered. Fixes [#150988](#150988) Fixes [#153478](#153478) ghstack-source-id: a8ece52 Pull Request resolved: #154142
|
Benchmarking failures appear to be present on |
desertfire
left a comment
There was a problem hiding this comment.
So what was the story about the previous memory leak?
@desertfire I was never able to definitively determine what happened, but the leak went away when we delayed codegen for fallback kernel arguments until the last possible second (since we don't need to codegen in some cases). I suspect the issue was that we codegen'ed the arguments, but then in some circumstances never deallocated them (since we didn't actually use them). |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Stack from ghstack (oldest at bottom):
Uses the new aoti_torch_call_dispatcher interface to call runtime fallback ops without calling back into Python. This supports a limited subset of input and output datatypes, but a significant majority of remaining fallback ATen ops are covered.
Fixes #150988
Fixes #153478
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov