Motivation

We want to have the option of having a more standard --moe-runner-backend flashinfer_cutedsl backend that is not specified to DeepEP. This PR integrates the Wrapper API exposed here: flashinfer-ai/flashinfer#2398.

Modifications

Server

• Add flashinfer_cutedsl as a modular moe_runner backend for --moe-runner-backend flashinfer_cutedsl with modelopt_fp4 quantization on the standard path (moe_a2a_backend=none). Implements the moe_runner pattern established by flashinfer_trtllm.py: a CuteDslFp4MoeQuantInfo dataclass carries weights/scales/wrapper, and a @register_fused_func("none", "flashinfer_cutedsl") handles FP4 quantization + CuteDslMoEWrapper.run(), aligning with the MoE refactor roadmap ([Roadmap] MoE Refactor #8715).
• Route through the generic FusedMoE -> StandardDispatcher -> MoeRunner pipeline instead of a standalone layer class, unblocking future moe_a2a_backend=flashinfer support.
• Support EP=1 (TP-sharded experts) and EP=TP (partitioned experts with all-reduce) configurations. CuteDSL's moe_sort handles EP with global expert IDs internally, so skip_local_expert_mapping is enabled.
• Enable FlashInfer autotuning for CuteDSL and switch the autotune warmup from torch.inference_mode() to torch.no_grad(), since CuteDSL lazily allocates persistent CUDA graph buffers during the first forward pass.
• Add FP4 weight preprocessing: post-quant W1 gate/up interleave and swizzled-to-MMA block-scale conversion during process_weights_after_loading.
• Add scale resolution logic (_resolve_cutedsl_standard_scales) that derives correct per-expert GEMM alphas from scalarized activation scales, handling EP slicing and multiple checkpoint formats.
• Preserve the existing DeepEP low-latency masked CuteDSL route (unchanged, already on main).

Tests

• test/registered/moe/test_cutedsl_moe.py: unit tests for wrapper accuracy vs. PyTorch reference, CUDA graph parity, and EP-sharded all-reduce correctness.
• test/registered/backends/test_deepseek_v3_fp4_cutedsl_moe.py: end-to-end GPQA accuracy on DeepSeek-V3 FP4 for both EP=1 and EP=4 configurations (nightly, 4 GPU B200).
• Verify no regressions on existing flashinfer_trtllm and flashinfer_cutlass backends.

Accuracy Tests

server (EP1):

python3 -m sglang.launch_server --model-path nvidia/DeepSeek-R1-0528-NVFP4-v2 --tensor-parallel-size=8 --cuda-graph-max-bs 256 --max-running-requests 256 --mem-fraction-static 0.85 --ep-size 1 --moe-runner-backend flashinfer_cutedsl --quantization modelopt_fp4

server (EP8):

python3 -m sglang.launch_server --model-path nvidia/DeepSeek-R1-0528-NVFP4-v2 --tensor-parallel-size=8 --cuda-graph-max-bs 256 --max-running-requests 256 --mem-fraction-static 0.85 --ep-size 8 --moe-runner-backend flashinfer_cutedsl --quantization modelopt_fp4

client:

python3 -m sglang.test.run_eval --port 30000 --eval-name gpqa --num-examples 198 --max-tokens 128000 --repeat 8 --thinking-mode deepseek-v3

results (EP1):

'scores': ['0.813', '0.818', '0.803', '0.823', '0.773', '0.798', '0.828', '0.763'], 'mean_score': np.float64(0.8023989898989901)

results (EP8):

'scores': ['0.788', '0.818', '0.778', '0.803', '0.793', '0.793', '0.808', '0.808'], 'mean_score': np.float64(0.798611111111111)

Benchmarking and Profiling

Generally this PR outperforms --moe-runner flashinfer_cutlass but not --moe-runner flashinfer_trtllm.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.