🐛 Describe the bug
Summary
In torchao, I have a unit test case that passes when ran individually, but fails with an inductor codgen error if the full set of test cases runs. Specifically, the Triton codegen references a variable that is undefined:
> if not (s85 == u31):
E NameError: name 's85' is not defined
/tmp/torchinductor_dev/xt/cxtidleizal7b2tytcnstzmeyst3gyouumtjkf57fv6jd3idnmyo.py:527: NameError
I tried clearing inductor cache, and the issue persists.
The test case in question to search for in the logs below is ScaleCalculationMode.FLOOR-emulated-True-True-True-1-16640-7168-2048
When I run the test case individually with TORCH_LOGS="output_code", it passes and i see 2 graphs compiled, aot forward 0 and backward 0: https://www.internalfb.com/phabricator/paste/view/P2223321552
When I run all cases TORCH_LOGS="output_code", I see compilation happen periodically, and I killed it at the first failure (scroll to bottom or search "ScaleCalculationMode.FLOOR-emulated-True-True-True-1-16640-7168-2048"). The most recent graph to compile is the one 5_backward, and that's the one with the undeclared variable s85: https://www.internalfb.com/phabricator/paste/view/P2223322938
Repro
Versions
cc @chauhang @penguinwu @ezyang @bobrenjc93 @aditvenk @laithsakka @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @aakhundov @coconutruben @jataylo
🐛 Describe the bug
Summary
In torchao, I have a unit test case that passes when ran individually, but fails with an inductor codgen error if the full set of test cases runs. Specifically, the Triton codegen references a variable that is undefined:
I tried clearing inductor cache, and the issue persists.
The test case in question to search for in the logs below is
ScaleCalculationMode.FLOOR-emulated-True-True-True-1-16640-7168-2048When I run the test case individually with TORCH_LOGS="output_code", it passes and i see 2 graphs compiled, aot forward 0 and backward 0: https://www.internalfb.com/phabricator/paste/view/P2223321552
When I run all cases TORCH_LOGS="output_code", I see compilation happen periodically, and I killed it at the first failure (scroll to bottom or search "
ScaleCalculationMode.FLOOR-emulated-True-True-True-1-16640-7168-2048"). The most recent graph to compile is the one 5_backward, and that's the one with the undeclared variable s85: https://www.internalfb.com/phabricator/paste/view/P2223322938Repro
2.12.0.dev20260305+cu130pytest test/prototype/moe_training/test_mxfp8_grouped_mm.py -v -s -k emulatedVersions
cc @chauhang @penguinwu @ezyang @bobrenjc93 @aditvenk @laithsakka @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @aakhundov @coconutruben @jataylo