Skip to content

[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604)#24542

Merged
simon-mo merged 31 commits intovllm-project:mainfrom
neuralmagic:luka/custom-pass-cleanup
Sep 22, 2025
Merged

Conversation

@ProExpertProg
Copy link
Copy Markdown
Collaborator

@ProExpertProg ProExpertProg commented Sep 9, 2025 •
edited by github-actions bot
Loading

Fixes #23091.

Purpose

Cleanup compilation tests and custom passes and address #23091 and part of #24376. This PR is a prerequisite for matching torch implementations of custom ops in #24604.

Cleanup:

  • RMSNorm + QuantFP8 fusion:
    • remove custom multi-output matching workaround
    • remove .instance() workaround
    • remove _seen_patterns.clear() workaround
    • rename FusionPass to RMSNormQuantFusionPass
  • reduce test_fusion.py running time
  • add cleanup pass that topo-sorts and does DCE
  • add LazyInitPass test utility
  • add env var for pattern match debugging (TODO: docs)
  • fix test_sequence_parallelism.py
  • add assert to test_silu_mul_quant_fusion for easier diagnostics of fp4 failures.
  • add init check for custom_ops field syntax and warnings for specifying ops that don't exist
  • add timing and graph dumping decorator for VllmInductorPass
  • add VllmPatternMatchPass common pass superclass with pattern tracking and dumping utilities

Test Plan

Unit tests all pass.

@mergify
Copy link
Copy Markdown

mergify bot commented Sep 9, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ProExpertProg.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Sep 9, 2025
@mergify mergify bot mentioned this pull request Sep 9, 2025
Closed
5 tasks
@ProExpertProg ProExpertProg force-pushed the luka/custom-pass-cleanup branch from ce05cd7 to 18c8d75 Compare September 9, 2025 22:52
@mergify mergify bot removed the needs-rebase label Sep 9, 2025
@ProExpertProg ProExpertProg force-pushed the luka/custom-pass-cleanup branch from 4bb59cd to 1778b80 Compare September 10, 2025 16:29
@ProExpertProg ProExpertProg marked this pull request as ready for review September 10, 2025 16:44
ProExpertProg
ProExpertProg commented Sep 10, 2025
View reviewed changes
@ProExpertProg
pass mgr util
955030c 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
remove old multi_output_match workaround, add new workaround (tests p…
d97634d 
…ass)

Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Improve workaround to properly handle inputs of inputs
c34cbdf 
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Remove prints, add debug statement
37446e5 
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Reduce fusion test running time
8d22102 
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Rename FusionPass, remove instance() workaround, other fusion pass cl…
7765aa8 
…eanup

Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Remove old QuantMultiOutputMatch
e8fae3d 
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Remove MultiOutputMatch workaround, add cleanup pass
4bdb73c 
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Add LazyInitPass
4f4cf01 
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Add pattern match debugging to tests
e584fcd 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
Fix seq-par test, add assert to silu-fp4 for clearer error
a5c2e3b 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
@ProExpertProg
more fusion.py cleanup
5fa85ea 
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
angelayi
angelayi approved these changes Sep 22, 2025
View reviewed changes
ProExpertProg
ProExpertProg commented Sep 22, 2025
View reviewed changes
ProExpertProg
ProExpertProg commented Sep 22, 2025
View reviewed changes
@ProExpertProg
Remove env
dba5449 
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
@ProExpertProg
Remove env
7fcbe74 
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
@ProExpertProg ProExpertProg enabled auto-merge (squash) September 22, 2025 16:48
@simon-mo simon-mo disabled auto-merge September 22, 2025 19:30
@simon-mo simon-mo merged commit d5e0fca into vllm-project:main Sep 22, 2025
41 of 43 checks passed
@github-project-automation github-project-automation bot moved this from In review to Done in torch.compile integration Sep 22, 2025
@ProExpertProg ProExpertProg deleted the luka/custom-pass-cleanup branch September 22, 2025 19:50
@ProExpertProg ProExpertProg mentioned this pull request Sep 22, 2025
Merged
@mgoin mgoin mentioned this pull request Sep 23, 2025
Closed
5 tasks
@ProExpertProg ProExpertProg mentioned this pull request Sep 23, 2025
Closed
1 task
@ProExpertProg ProExpertProg mentioned this pull request Sep 24, 2025
Closed
1 task
ABC12345anouys pushed a commit to ABC12345anouys/vllm that referenced this pull request Sep 25, 2025
@ProExpertProg @ABC12345anouys
[torch.compile] Cleanup compilation tests and custom passes, add debu…
14b3afe 
…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542)

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
ZJY0516
ZJY0516 reviewed Sep 25, 2025
View reviewed changes
vllm/compilation/vllm_inductor_pass.py
debug_dump_path = Path(debug_dump_path) / f"rank_{rank}"
debug_dump_path.mkdir(parents=True, exist_ok=True)

from vllm.utils import unique_filepath
Copy link
Copy Markdown
Member

@ZJY0516 ZJY0516 Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that vllm.utils does not have unique_filepath.

Copy link
Copy Markdown
Collaborator Author

@ProExpertProg ProExpertProg Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, my bad, was working on a newer branch. I'll fix in #24604 unless you want to extract the change from there (and pull the import back up) in a separate PR to fix earlier

Copy link
Copy Markdown
Member

@ZJY0516 ZJY0516 Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix it in a separate PR

charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
@ProExpertProg @charlifu
[torch.compile] Cleanup compilation tests and custom passes, add debu…
1fe97c4 
…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542)

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
@ZJY0516 ZJY0516 mentioned this pull request Sep 26, 2025
Merged
5 tasks
yewentao256 pushed a commit that referenced this pull request Oct 3, 2025
@ProExpertProg @yewentao256
[torch.compile] Cleanup compilation tests and custom passes, add debu…
6dbbecd 
…g utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542)

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
@ProExpertProg @choprahetarth
[torch.compile] Cleanup compilation tests and custom passes, add debu…
91d1f53 
…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542)

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@ProExpertProg @lywa1998
[torch.compile] Cleanup compilation tests and custom passes, add debu…
fd013d3 
…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542)

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
@ProExpertProg @rtourgeman
[torch.compile] Cleanup compilation tests and custom passes, add debu…
f80bfa7 
…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542)

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: luka <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ZJY0516 ZJY0516 ZJY0516 left review comments

@zou3519 zou3519 zou3519 approved these changes

@simon-mo simon-mo Awaiting requested review from simon-mo

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

@houseroad houseroad Awaiting requested review from houseroad houseroad is a code owner

@hmellor hmellor Awaiting requested review from hmellor hmellor is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 is a code owner

@ilmarkov ilmarkov Awaiting requested review from ilmarkov

+3 more reviewers

@nvpohanh nvpohanh nvpohanh left review comments

@elvischenv elvischenv elvischenv left review comments

@angelayi angelayi angelayi approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ready ONLY add when PR is ready to merge/full CI is needed torch.compile

Projects

torch.compile integration
Status: Done

Milestone

torch==2.9.0 compilation improvements

8 participants

@ProExpertProg @zou3519 @ilmarkov @angelayi @nvpohanh @ZJY0516 @elvischenv @simon-mo