Summary of Changes

Hello @ClawSeven, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request establishes a new Continuous Integration (CI) pipeline for diffusion Large Language Models (LLMs). It integrates testing for the LLaDA2.0-mini model, configuring it to use the LowConfidence DLLM algorithm. The CI setup includes comprehensive performance benchmarks, evaluating both accuracy on the gsm8k dataset and single-batch inference speed, ensuring robust validation of diffusion LLM capabilities.

Highlights

• New CI for Diffusion LLMs: This pull request introduces a new Continuous Integration (CI) setup specifically designed for diffusion Large Language Models (LLMs), expanding the project's testing capabilities.
• LLaDA2.0-mini Model Integration: A new test suite has been added to validate the LLaDA2.0-mini model, including its server setup and performance characteristics.
• LowConfidence DLLM Algorithm Configuration: A configuration file (low_confidence.yaml) and corresponding server argument (--dllm-algorithm LowConfidence) have been added to enable testing with the LowConfidence diffusion LLM algorithm.
• Performance Benchmarking: The new CI includes benchmarks for the gsm8k dataset to assess accuracy and output throughput, as well as a single-batch speed test to measure inference performance.

[Performance in tp=1, L40]

This Llada2- mini model is small.
Even with this setting, we can test:
tp=1
NVIDIA L40
bs=1

[CI Test Method] TestLLaDA2Mini.test_bs_1_speed
+-------------+--------+------------+-----------------+
| Latency (s) | Tokens | Acc Length | Speed (token/s) |
+-------------+--------+------------+-----------------+
|    6.038    |  870   |   1.000    |     144.09      |
+-------------+--------+------------+-----------------+
speed=144.09

[CI Test Method] TestLLaDA2Mini.test_gsm8k

Accuracy: 0.925
Invalid: 0.000
Latency: 361.234 s
Output throughput: 67.848 token/s

[Performance in tp=4, A100]
To be added later....

And consider adding more test to nightly-test after this, for performance monitoring

Based on the current test cases, it looks like the nightly tests have the performance tests in large models like DeepSeek-V3. We may add a large model like llada2-flash 100B into this in the future.

Currently, we already have the performance test in llada2-mini, which should be enough for this PR. @ClawSeven

/tag-and-rerun-ci

This PR looks clean. Great job.

The current CI for llada2-mini is in the setting of 4GPUs.
The ci is skipped here: https://github.com/sgl-project/sglang/actions/runs/20286966160/job/58286368592?pr=14723

We may add it to the 2-GPU setting, as the model is super small.

TP2 / H20

Accuracy: 0.930
Invalid: 0.000
Latency: 101.512 s
Output throughput: 248.148 token/s
metrics={'accuracy': 0.93, 'invalid': 0.0, 'latency': 101.51187389157712, 'output_throughput': 248.14831048144134}

Let me know if you'd like to add databases, authentication, or Docker!

+-------------+--------+------------+-----------------+
| Latency (s) | Tokens | Acc Length | Speed (token/s) |
+-------------+--------+------------+-----------------+
| 1.763 | 688 | 1.000 | 390.17 |
+-------------+--------+------------+-----------------+
speed=390.17
.

Ran 1 test in 32.810s

TP1

Accuracy: 0.930
Invalid: 0.000
Latency: 130.141 s
Output throughput: 190.724 token/s
metrics={'accuracy': 0.93, 'invalid': 0.0, 'latency': 130.14087017811835, 'output_throughput': 190.7240974032872}
.
----------------------------------------------------------------------
Ran 1 test in 431.747s

OK

---

Let me know if you'd like to add authentication, SQLite, or database integration!

+-------------+--------+------------+-----------------+
| Latency (s) | Tokens | Acc Length | Speed (token/s) |
+-------------+--------+------------+-----------------+
|    2.187    |  672   |   1.000    |     307.21      |
+-------------+--------+------------+-----------------+
speed=307.21
.
----------------------------------------------------------------------
Ran 1 test in 33.225s

OK

/rerun-failed-ci

/rerun-failed-ci try again