[inductor] move max-autotune logic inside V.choices.get_mm_configs by coconutruben · Pull Request #161344 · pytorch/pytorch

coconutruben · 2025-08-23T02:57:52Z

Stack from ghstack (oldest at bottom):

why

heuristics providers know decide whether to (or which choices to add)
in the max-autotune case
enables an eventual override point to gracefully fallback to the
standard behavior

what

max-autotune is determined inside V.choices.get_mm_configs
because it's mm only right now, we can just do
config.max_autotune or config.max_autotune_gemm
a TODO indicates that this can change in the future when this
expands to more templates

testing

python3 -bb -m pytest test/inductor/test_max_autotune.py -v

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

Differential Revision: D81520573

\# why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior \# what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` [ghstack-poisoned]

pytorch-bot · 2025-08-23T02:57:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161344

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit c6813ed with merge base 5da573c ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…_configs" \# why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior \# what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

# why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: #161346 Approved by: https://github.com/eellison ghstack dependencies: #162075, #161340, #161341, #161342, #161343, #161344, #161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: #161347 Approved by: https://github.com/eellison ghstack dependencies: #162075, #161340, #161341, #161342, #161343, #161344, #161345, #161346

…#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: #161348 Approved by: https://github.com/eellison ghstack dependencies: #162075, #161340, #161341, #161342, #161343, #161344, #161345, #161346, #161347

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

\# why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior \# what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` ghstack-source-id: e43a4da Pull Request resolved: pytorch/pytorch#161344

pytorch-bot Bot added ciflow/inductor module: inductor labels Aug 23, 2025

coconutruben added the topic: not user facing topic category label Aug 23, 2025

coconutruben added 2 commits August 22, 2025 20:05

github-actions Bot deleted the gh/coconutruben/45/head branch October 6, 2025 02:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[inductor] move max-autotune logic inside V.choices.get_mm_configs#161344

[inductor] move max-autotune logic inside V.choices.get_mm_configs#161344
coconutruben wants to merge 29 commits intogh/coconutruben/45/basefrom
gh/coconutruben/45/head

coconutruben commented Aug 23, 2025 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Aug 23, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

coconutruben commented Aug 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

why

what

testing

Uh oh!

pytorch-bot Bot commented Aug 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161344

✅ No Failures

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coconutruben commented Aug 23, 2025 •

edited

Loading

pytorch-bot Bot commented Aug 23, 2025 •

edited

Loading