Skip to content

[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registration#32855

Merged
mgoin merged 2 commits into
vllm-project:mainfrom
fadara01:fix_op_registration
Jan 22, 2026
Merged

[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registration#32855
mgoin merged 2 commits into
vllm-project:mainfrom
fadara01:fix_op_registration

Conversation

@fadara01

@fadara01 fadara01 commented Jan 22, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

[BugFix] Fix invalid numeric default value in flashinfer_fused_moe_blockscale_fp8 op registration

Fixes: #32840

Purpose

Fix invalid numeric default value in flashinfer_fused_moe_blockscale_fp8 op registration
The default value has to be an int an not an enum as the error indicates:

invalid numeric default value:
flashinfer_fused_moe_blockscale_fp8(Tensor routing_logits, Tensor routing_bias, Tensor x, Tensor w13_weight, Tensor w13_weight_scale_inv, Tensor w2_weight, Tensor w2_weight_scale_inv, SymInt global_num_experts, SymInt top_k, SymInt? num_expert_group, SymInt? topk_group, SymInt intermediate_size, SymInt expert_offset, SymInt local_num_experts, SymInt[] block_shape, SymInt routing_method_type=RoutingMethodType.DeepSeekV3, float? routed_scaling=1.0) -> Tensor                                                                                                                                                                                                                                                                                                                                                                                                                        ~ <--- HERE

Test Plan

Reproducer on #32840

Test Result

Reproducer doesn't fail

Essential Elements of an Effective PR Description Checklist
  • [ Y] The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • [ Y] The test plan, such as providing test command.
  • [ Y] The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@mergify mergify Bot added nvidia bug Something isn't working labels Jan 22, 2026
@fadara01 fadara01 mentioned this pull request Jan 22, 2026
Merged
5 tasks
gemini-code-assist[bot]
gemini-code-assist Bot reviewed Jan 22, 2026
View reviewed changes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly fixes a bug in the registration of the flashinfer_fused_moe_blockscale_fp8 custom operator. The issue was an "invalid numeric default value" error caused by using an enum member as a default value for an integer parameter. The fix addresses this by using the .value attribute to provide the integer value from the enum. A related change in a call site ensures consistency by also comparing against the enum's integer value. The changes are accurate and resolve the bug.

@mergify

mergify Bot commented Jan 22, 2026

Copy link
Copy Markdown
Contributor

Hi @fadara01, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally: 
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@fadara01

fadara01 commented Jan 22, 2026

Copy link
Copy Markdown
Contributor Author

Could you please take a look? @mgoin / @bigPYJ1151 / @robertgshaw2-redhat

@fadara01
[BugFix] Fix invalid numeric default value in flashinfer_fused_moe_bl…
eb5a933 
…ockscale_fp8 op registration

Default value has to be int and not enum

Fixes: vllm-project#32840

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
@fadara01 fadara01 force-pushed the fix_op_registration branch from a4abd21 to eb5a933 Compare January 22, 2026 13:20
@fadara01 fadara01 changed the title [BugFix] Fix invalid numeric default value in flashinfer_fused_moe_blockscale_fp8 op registration [BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registration Jan 22, 2026
mgoin
mgoin approved these changes Jan 22, 2026
View reviewed changes

@mgoin mgoin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me, thanks for the fix

@github-project-automation github-project-automation Bot moved this to Ready in NVIDIA Jan 22, 2026
@mgoin mgoin enabled auto-merge (squash) January 22, 2026 13:55
@github-actions github-actions Bot added the ready ONLY add when PR is ready to merge/full CI is needed label Jan 22, 2026
yewentao256
yewentao256 approved these changes Jan 22, 2026
View reviewed changes

@yewentao256 yewentao256 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the work!

@fadara01

fadara01 commented Jan 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing!
The CI failures here are not related, should we force merge?

@mgoin mgoin merged commit fc56f4a into vllm-project:main Jan 22, 2026
52 checks passed
@github-project-automation github-project-automation Bot moved this from Ready to Done in NVIDIA Jan 22, 2026
cwazai pushed a commit to cwazai/vllm that referenced this pull request Jan 25, 2026
@fadara01 @cwazai
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registrat…
0f4fd95 
…ion (vllm-project#32855)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Signed-off-by: 陈建华 <1647430658@qq.com>
lapy pushed a commit to lapy/vllm that referenced this pull request Jan 27, 2026
@fadara01 @lapy
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registrat…
3dd9d47 
…ion (vllm-project#32855)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
mystous pushed a commit to mystous/vllm_hybrid that referenced this pull request May 10, 2026
@fadara01
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registrat…
0d53ec5 
…ion (vllm-project#32855)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
@fadara01
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registrat…
a75894e 
…ion (vllm-project#32855)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
my-other-github-account pushed a commit to my-other-github-account/vllm that referenced this pull request May 15, 2026
@fadara01
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registrat…
d35d017 
…ion (vllm-project#32855)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
0826joyce pushed a commit to 0826joyce/vllm-serving-optimization that referenced this pull request May 19, 2026
@fadara01
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registrat…
8468a38 
…ion (vllm-project#32855)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mgoin mgoin mgoin approved these changes

@yewentao256 yewentao256 yewentao256 approved these changes

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth

@pavanimajety pavanimajety Awaiting requested review from pavanimajety pavanimajety is a code owner

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

bug Something isn't working nvidia ready ONLY add when PR is ready to merge/full CI is needed

Projects

NVIDIA
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: [CPU Backend] Engine crashed due to error on flashinfer op registration

3 participants

@fadara01 @mgoin @yewentao256