Motivation

The SM120 fp8 blockwise GEMM kernel was using KernelScheduleAuto as the schedule, which on SM120 happens to select the cooperative kernel only. The single-kernel approach misses a performance opportunity, the pingpong schedule is about 2x faster than cooperative for small M. I adapted the example from the CUTLASS repo.

Modifications

sgl-kernel/csrc/gemm/fp8_blockwise_gemm_kernel.cu:

• Replaced KernelScheduleAuto with KernelScheduleSm120Blockwise for the cooperative path, when M > 64, to avoid
  the specific CUTLASS issue (the refcheck output will explode in relative error, I'm not sure of the cause exactly, but it appears to be potentially a CUTLASS library issue)
• Added a pingpong path using KernelTmaWarpSpecializedBlockwisePingpongSm120 with a 64x128x128 tile shape for M ≤ 64.
• Added a m <= 64 runtime check to pick between the two paths.
• Refactor the kernel setup a bit.

Accuracy and UT

RTX 5090 (SM120) against Flashinfer across Qwen/Qwen3.5-27B-FP8 shapes at M from 1 to 512:

Performance (RTX 5090, N=1536, K=5120)

Profiles:

E2E Accuracy:

python -m sglang.test.run_eval --base-url http://localhost:30000 --eval-name gsm8k --num-examples 200 --max-tokens 16000 --repeat 5 --num-threads 48 --num-shots 5 --temperature 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --chat-template-kwargs '{"enable_thinking": true}'

Before:
{'score:std': np.float64(0.07053367989832945), 'scores': ['0.995', '0.975', '0.980', '0.990', '0.995'], 'mean_score': np.float64(0.9870000000000001)}

After:
{'score:std': np.float64(0.12155245781143219), 'scores': ['0.990', '0.985', '0.995', '0.995', '0.985'], 'mean_score': np.float64(0.99)}

BS = 1 speed:

Before:
(It will use the cooperative schedule)

+-------------+--------+------------+-----------------+
| Latency (s) | Tokens | Acc Length | Speed (token/s) |
+-------------+--------+------------+-----------------+
|   29.996    |  1024  |   1.000    |      34.14      |
+-------------+--------+------------+-----------------+

After:

(I only zoom in really far, to show the ping-pong schedule name)

+-------------+--------+------------+-----------------+
| Latency (s) | Tokens | Acc Length | Speed (token/s) |
+-------------+--------+------------+-----------------+
|   19.652    |  1024  |   1.000    |      52.11      |
+-------------+--------+------------+-----------------+

I think we can change the default GEMM backend on SM120 later.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.