Motivation

Dynamic batching in DLLM was added in PR #14883, and the cuda graph batch size limitation should have been removed then, but it was overlooked. This PR removes the limitation.

Modifications

Remove hard-coded override of cuda_graph_bs for DLLM.

Accuracy Tests

Tested on H20-3e using test/registered/dllm/test_dllm_batching.py

Before this PR:

Accuracy: 0.915
Invalid: 0.000
Latency: 240.785 s
Output throughput: 102.216 token/s
metrics={'accuracy': 0.915, 'invalid': 0.0, 'latency': 240.7848315560259, 'output_throughput': 102.21574108696822}

After this PR:

Accuracy: 0.915
Invalid: 0.000
Latency: 130.954 s
Output throughput: 187.485 token/s
metrics={'accuracy': 0.915, 'invalid': 0.0, 'latency': 130.9542715421412, 'output_throughput': 187.4852932315319}

Benchmarking and Profiling

Please refer to the output-throughput results in the "Accuracy Tests" above.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.