[inductor] support linear & layer_norm unbacked#155267
[inductor] support linear & layer_norm unbacked#155267ColinPeppler wants to merge 5 commits intogh/ColinPeppler/72/basefrom
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155267
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ You can merge normally! (1 Unrelated Failure)As of commit b37ff27 with merge base 25fbf09 ( UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
We discussed this at composability sync. The conclusion was:
|
### What - Use `statically_known_true` over `guard_size_oblivious` in cases where we're checking an optimization path. Otherwise, it will DDE and we can't take the safe/slower path. - For broadcast checks, use fallback=False if we encounter a DDE. Typically, unbackeds would be ≥2 and that falls inline with size-oblivious reasoning. - In `coerce_tangent_and_suggest_memory_format`, use the torch._refs version of `contiguous` over Aten's version because Aten doesn't have a DDE-friendly path (yet). ### Example DDE ``` GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(128*((u0//387)), 0) (unhinted: Eq(128*((u0//387)), 0)). (Size-like symbols: u0) Caused by: (_functorch/_aot_autograd/collect_metadata_analysis.py:84 in coerce_tangent_and_suggest_memory_format) ``` ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/lowering.py:488 in broadcast_symbolic_shapes) ``` ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/ir.py:2797 in create) ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]
### What - Use `statically_known_true` over `guard_size_oblivious` in cases where we're checking an optimization path. Otherwise, it will DDE and we can't take the safe/slower path. - For broadcast checks, use fallback=False if we encounter a DDE. Typically, unbackeds would be ≥2 and that falls inline with size-oblivious reasoning. - In `coerce_tangent_and_suggest_memory_format`, use the torch._refs version of `contiguous` over Aten's version because Aten doesn't have a DDE-friendly path (yet). ### Example DDE ``` GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(128*((u0//387)), 0) (unhinted: Eq(128*((u0//387)), 0)). (Size-like symbols: u0) Caused by: (_functorch/_aot_autograd/collect_metadata_analysis.py:84 in coerce_tangent_and_suggest_memory_format) ``` ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/lowering.py:488 in broadcast_symbolic_shapes) ``` ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/ir.py:2797 in create) ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]
|
@eellison the old used to used size_oblivious=True anyway so this is not changing existing behaviour for unbacked anwyay size_oblivious=True means assume unbacked>=2 |
torch/_inductor/lowering.py
Outdated
| ( | ||
| V.graph.sizevars.shape_env.evaluate_expr( | ||
| sympy.Eq(a, 1), size_oblivious=True | ||
| sympy.Eq(a, 1), size_oblivious=True, fallback_value=False |
There was a problem hiding this comment.
remove size_oblivious=True ditto for others.
### What - Use `statically_known_true` over `guard_size_oblivious` in cases where we're checking an optimization path. Otherwise, it will DDE and we can't take the safe/slower path. - For broadcast checks, use fallback=False if we encounter a DDE. Typically, unbackeds would be ≥2 and that falls inline with size-oblivious reasoning. ### Example DDE ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/lowering.py:488 in broadcast_symbolic_shapes) ``` ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/ir.py:2797 in create) ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]
torch/_inductor/lowering.py
Outdated
| V.graph.sizevars.shape_env.evaluate_expr( | ||
| sympy.Eq(a, 1), size_oblivious=True | ||
| sympy.Eq(a, 1), fallback_value=False | ||
| ) | ||
| and not V.graph.sizevars.shape_env.evaluate_expr( | ||
| sympy.Eq(b, 1), size_oblivious=True | ||
| sympy.Eq(b, 1), fallback_value=False | ||
| ) | ||
| ) | ||
| or ( | ||
| not V.graph.sizevars.shape_env.evaluate_expr( | ||
| sympy.Eq(a, 1), size_oblivious=True | ||
| sympy.Eq(a, 1), fallback_value=False | ||
| ) | ||
| and V.graph.sizevars.shape_env.evaluate_expr( | ||
| sympy.Eq(b, 1), size_oblivious=True | ||
| sympy.Eq(b, 1), fallback_value=False | ||
| ) |
There was a problem hiding this comment.
Can we reuse the above broadcast_symbolic_shapes here instead of duplicating the logic ? (i know this was preexisting to your change)
There was a problem hiding this comment.
this one would look a lil bit forced since broadcast_symbolic_shapes returns the broadcasted shape vs. here we just want to check for broadcast.
But agreed the code looks duplicated, I'm gonna find a more reusable way.
### What - Use `statically_known_true` over `guard_size_oblivious` in cases where we're checking an optimization path. Otherwise, it will DDE and we can't take the safe/slower path. - For broadcast checks, use `fallback=False` if we encounter a DDE. Typically, unbackeds would be ≥2 and that falls inline with size-oblivious reasoning (i.e. when `size_oblivious=True`). ### Example DDE ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/lowering.py:488 in broadcast_symbolic_shapes) ``` ``` torch._inductor.exc.InductorError: LoweringException: GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq((u0//387), 1) (unhinted: Eq((u0//387), 1)). (Size-like symbols: u0) Caused by: (_inductor/ir.py:2797 in create) ``` cc ezyang penguinwu bobrenjc93 voznesenskym EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
What
statically_known_trueoverguard_size_obliviousin cases where we're checking an optimization path. Otherwise, it will DDE and we can't take the safe/slower path.fallback=Falseif we encounter a DDE. Typically, unbackeds would be ≥2 and that falls inline with size-oblivious reasoning (i.e. whensize_oblivious=True).Example DDE
Stack from ghstack (oldest at bottom):
cc @ezyang @penguinwu @bobrenjc93 @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben