Skip to content

[hoo] Invoke subgraph + effects Inductor support#167364

Closed
angelayi wants to merge 14 commits intogh/angelayi/135/basefrom
gh/angelayi/135/head
Closed

[hoo] Invoke subgraph + effects Inductor support#167364
angelayi wants to merge 14 commits intogh/angelayi/135/basefrom
gh/angelayi/135/head

Conversation

@angelayi
Copy link
Contributor

@angelayi angelayi commented Nov 7, 2025

In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an ir.EffectfulKernel, and the op is treated as a FallbackKernel. However in the case of invoke_subgraph, we want it to be an ir.InvokeSubgraph to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an ir.EffectfulKernel, I just add this dependency information to V.graph.additional_star_deps, similar to V.graph.additional_buffer_deps which is used by control_deps.

Stack from ghstack (oldest at bottom):

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo @chenyang78

Differential Revision: D87668982

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 7, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/167364

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 2 Unrelated Failures

As of commit 5df883f with merge base e3f24fd (image):

UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
angelayi added a commit that referenced this pull request Nov 7, 2025
ghstack-source-id: 252e703
Pull Request resolved: #167364
@eellison eellison removed their request for review November 11, 2025 19:31
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
angelayi added a commit that referenced this pull request Nov 12, 2025
ghstack-source-id: 186e7c1
Pull Request resolved: #167364
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
This was referenced Nov 14, 2025
@angelayi angelayi added the topic: not user facing topic category label Nov 14, 2025
@fxdawnn
Copy link
Contributor

fxdawnn commented Nov 15, 2025

follow for update

Khanaksahu pushed a commit to Khanaksahu/pytorch that referenced this pull request Nov 17, 2025
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
angelayi added a commit that referenced this pull request Nov 19, 2025
ghstack-source-id: f9f8635
Pull Request resolved: #167364
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

[ghstack-poisoned]
angelayi added a commit that referenced this pull request Nov 21, 2025
ghstack-source-id: 6e84acc
Pull Request resolved: #167364
@angelayi
Copy link
Contributor Author

@angelayi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 21, 2025
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)

[ghstack-poisoned]
@angelayi
Copy link
Contributor Author

@angelayi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)

[ghstack-poisoned]
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)

[ghstack-poisoned]
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)

[ghstack-poisoned]
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)

[ghstack-poisoned]
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`. 




cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)

[ghstack-poisoned]
@angelayi
Copy link
Contributor Author

angelayi commented Dec 3, 2025

@angelayi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@angelayi
Copy link
Contributor Author

angelayi commented Dec 3, 2025

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

pytorchmergebot pushed a commit that referenced this pull request Dec 3, 2025
We shouldn't need this anymore as we have a registration for the op to have no effect

Differential Revision: [D87680134](https://our.internmc.facebook.com/intern/diff/D87680134)

Pull Request resolved: #168389
Approved by: https://github.com/zou3519
ghstack dependencies: #167364
pytorchmergebot pushed a commit that referenced this pull request Dec 4, 2025
We shouldn't need this anymore as we have a registration for the op to have no effect

Differential Revision: [D87680134](https://our.internmc.facebook.com/intern/diff/D87680134)

Pull Request resolved: #168389
Approved by: https://github.com/zou3519
ghstack dependencies: #167364
JacobSzwejbka pushed a commit that referenced this pull request Dec 8, 2025
In order to support effect tokens with invoke subgraph in Inductor, I reimplemented how we handle with_effects, which now has a similar behavior as control_deps.

Previously the behavior of with_effects is that we turn each with_effects(op, ..) call into an `ir.EffectfulKernel`, and the `op` is treated as a FallbackKernel. However in the case of `invoke_subgraph`, we want it to be an `ir.InvokeSubgraph` to trigger the invoke_subgraph inductor lowering logic. So now instead of turning with_effects into an `ir.EffectfulKernel`, I just add this dependency information to `V.graph.additional_star_deps`, similar to `V.graph.additional_buffer_deps` which is used by `control_deps`.

Differential Revision: [D87668982](https://our.internmc.facebook.com/intern/diff/D87668982)
Pull Request resolved: #167364
Approved by: https://github.com/fxdawnn
JacobSzwejbka pushed a commit that referenced this pull request Dec 8, 2025
We shouldn't need this anymore as we have a registration for the op to have no effect

Differential Revision: [D87680134](https://our.internmc.facebook.com/intern/diff/D87680134)

Pull Request resolved: #168389
Approved by: https://github.com/zou3519
ghstack dependencies: #167364
JacobSzwejbka pushed a commit that referenced this pull request Dec 8, 2025
We shouldn't need this anymore as we have a registration for the op to have no effect

Differential Revision: [D87680134](https://our.internmc.facebook.com/intern/diff/D87680134)

Pull Request resolved: #168389
Approved by: https://github.com/zou3519
ghstack dependencies: #167364
tiendatngcs pushed a commit to tiendatngcs/pytorch-Dec25 that referenced this pull request Dec 10, 2025
tiendatngcs pushed a commit to tiendatngcs/pytorch-Dec25 that referenced this pull request Dec 10, 2025
@github-actions github-actions bot deleted the gh/angelayi/135/head branch January 3, 2026 02:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants