[Model Runner V2] Support stock torch compile for v2#41667
Conversation
Signed-off-by: yewentao256 <zhyanwentao@126.com>
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
|
Hi @yewentao256, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
There was a problem hiding this comment.
Code Review
This pull request introduces support for STOCK_TORCH_COMPILE within the GPU model runner, including an environment patch and model compilation logic. A critical issue was identified where the compilation call is incorrectly implemented as an in-place method on the model rather than using torch.compile and capturing the returned optimized module. Additionally, it is recommended to move the compilation block earlier in the initialization sequence so that dependent components utilize the compiled version of the model.
|
Would like to get @WoosukKwon's opinion on this one |
|
@yewentao256 test failures are related - looks like some test mocks need updating |
yewentao256
left a comment
There was a problem hiding this comment.
@njhill Thanks! FIxed
|
From discussion with @WoosukKwon we may not add this to MRV2 for now. @yewentao256 has update the Oracle logic to reflect this. |
|
Close this PR as we are not going to support stock torch compile for v2 |
Purpose
Support stock torch compile for v2
Part of the #41286
Originally
Now
======================== 1 passed, 38 deselected, 22 warnings in 9.56s ========================