sync main by tc-mb · Pull Request #1 · tc-mb/sglang

tc-mb · 2025-08-06T08:21:34Z

Motivation

Modifications

Accuracy Test

Benchmark & Profiling

Checklist

Format your code according to the Code Formatting with Pre-Commit.
Add unit tests as outlined in the Running Unit Tests.
Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

…iggering `torch.compile` in forward pass (sgl-project#8353)

Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>

…Instruct (sgl-project#8344) Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>

…in Qwen3 detector (sgl-project#8357)

…gl-project#8363)

Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com> Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>

…8333)

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com> Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

…d fix torch reduce (sgl-project#8267) Co-authored-by: CuiBo 82354186+SuperCB@users.noreply.github.com Co-authored-by: GeLee 865038696@qq.com Co-authored-by: 杨睿 yangruipis@163.com

Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

)

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com> Co-authored-by: zxy <zhou0493@e.ntu.edu.sg> Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com> Co-authored-by: Mick <mickjagger19@icloud.com> Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

…gl-project#8396)

Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>

…t#8776)

Co-authored-by: wunhuang <wunhuang@amd.com> Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>

…loat32 (sgl-project#8212) Co-authored-by: jianan-gu <jianan.gu@intel.com> Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>

…ject#8571) Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>

…gl-project#8744)

…-project#8797)

Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>

Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>

…project#8681) Co-authored-by: wunhuang <wunhuang@amd.com>

…head (sgl-project#8751)

…n update expert weights (sgl-project#8753)

Co-authored-by: Co-author fzyzcjy <ch271828n@outlook.com> Co-authored-by: fzyzcjy <5236035+fzyzcjy@users.noreply.github.com> Co-authored-by: zhuofan1123 <zhuofanl@nvidia.com> Co-authored-by: liz-badada <jinyanc@nvidia.com> Co-authored-by: xutizhou <xutingz@nvidia.com> Co-authored-by: linhu-nv <linhu@nvidia.com>

BBuf and others added 30 commits July 25, 2025 21:17

[torch.compile bug] avoid biased_grouped_topk_impl func repeatedly tr…

9045cc1

…iggering `torch.compile` in forward pass (sgl-project#8353)

[P/D] Support ipv6 in P/D scenario (sgl-project#7858)

1b9cea5

Co-authored-by: Shangming Cai <caishangming@linux.alibaba.com>

Add H20-3e fused MoE kernel tuning configs for Qwen3-Coder-480B-A35B-…

12cb760

…Instruct (sgl-project#8344) Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>

[Bugfix][Feat] Add XML-ish grammar in EBNFComposer and fix misc bugs …

f8260f2

…in Qwen3 detector (sgl-project#8357)

Clean up server_args, triton cache manager (sgl-project#8332)

ed2e313

fix: upgrade nccl version (sgl-project#8359)

7181ec8

[Feat] Add reasoning parser for Qwen/Qwen3-235B-A22B-Thinking-2507 (s…

d8ee156

…gl-project#8363)

fix: kimi k2 xgrammar crash (sgl-project#8367)

f8ca236

Co-authored-by: cicirori <32845984+cicirori@users.noreply.github.com> Co-authored-by: gongwei-130 <56567052+gongwei-130@users.noreply.github.com>

Fix FP4 MoE accuracy from missing routed_scaling_factor (sgl-project#…

58c468f

…8333)

[CI] Fix flaky threshold (sgl-project#8370)

3ec0b21

chore: bump v0.4.9.post4 (sgl-project#8305)

2272c2a

Fix test_moe_fused_gate_combined sgl-kernel ci test (sgl-project#8374)

8af145b

Uodate Dockerfile.gb200 to latest sglang (sgl-project#8356)

e6312d2

chore: improve mmmu benchmark (sgl-project#7000)

4fa44d6

Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com> Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

Save peak memory in logits processor (sgl-project#8343)

e236d8f

Extract update_weights from RL Engine to SGLang to keep simplicity an…

ce32bc2

…d fix torch reduce (sgl-project#8267) Co-authored-by: CuiBo 82354186+SuperCB@users.noreply.github.com Co-authored-by: GeLee 865038696@qq.com Co-authored-by: 杨睿 yangruipis@163.com

chore: improvements on mm_utils (sgl-project#7737)

5347567

vlm: optimize tensor transport (sgl-project#6003)

3212c2a

Co-authored-by: Xinyuan Tong <115166877+JustinTong0323@users.noreply.github.com>

Tiny assert EPLB is used together with expert parallel (sgl-project#8381

da0c026

)

Add perf tests for LoRA (sgl-project#8314)

5c705b1

Remove slot usage in code to be backward-compatible with python 3.9 (s…

7615463

…gl-project#8396)

Add docker release flow for gb200 (sgl-project#8394)

62a6b7c

HiCache, check before terminate prefetching (sgl-project#8372)

528bd1e

Add nvfp4 scaled mm benchmark. (sgl-project#8401)

426b749

Urgent Fix: intern-s1 chat-template matching (sgl-project#8403)

b602f42

Signed-off-by: Xinyuan Tong <justinning0323@outlook.com>

Tool to dump and compare internal activation tensors (sgl-project#7976)

ed0fdbf

Minor tool for comparison of benchmark results (sgl-project#7974)

62222bd

Fix bench script making input data on L2 cache (sgl-project#7739)

e34cf6a

[NVIDIA] Add Flashinfer MoE blockscale fp8 backend (sgl-project#8036)

85486b6

lifuhuang and others added 29 commits August 4, 2025 17:11

[Fix] Fix several issues preventing gemma3n LoRA support. (sgl-projec…

7cb2075

…t#8776)

Support OCP MXFP4 quantization on AMD GPUs (sgl-project#8255)

d4bf5a8

Co-authored-by: wunhuang <wunhuang@amd.com> Co-authored-by: Hubert Lu <Hubert.Lu@amd.com>

[CPU][sgl-kernel] biased_grouped_topk: fix correction_bias dtype to f…

08f8f49

…loat32 (sgl-project#8212) Co-authored-by: jianan-gu <jianan.gu@intel.com> Co-authored-by: YanbingJiang <yanbing.jiang@intel.com>

[PD] Refactor parallel sizes and add pp support for mooncake (sgl-pro…

d98a491

…ject#8571) Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>

[pd-router] Add Configurable Retry Logic for reduce backend pressure (s…

354ac43

…gl-project#8744)

chore: upgrade flashinfer v0.2.9 (sgl-project#8780)

1ea94d3

[NVIDIA]Fix local_num_experts for EP (sgl-project#8779)

b01eeb8

[feat] Add detail in image_data (sgl-project#8596)

873f384

Revert "[NVIDIA]Fix local_num_experts for EP (sgl-project#8779)" (sgl…

5e91fed

…-project#8797)

feat: support sgl-kernel cu129 (sgl-project#8800)

194561f

chore: bump sgl-kernel v0.3.2 (sgl-project#8802)

75df31b

feat: add trtllm-gen mha from direct call (sgl-project#8782)

40e3b2b

Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>

GLM-4.5 and GLM-4.5-Air both support (sgl-project#8804)

a4b0d5c

fix: update cmake (sgl-project#8817)

8e8545c

chore: upgrade transformers 4.55.0 (sgl-project#8823)

901ab75

Co-authored-by: hebiao064 <hebiaobuaa@gmail.com>

chore: upgrade flashinfer 0.2.10 (sgl-project#8827)

4f4e0e4

Fix potential memory fault issue and ncclSystemError in CI test (sgl-…

32d9e39

…project#8681) Co-authored-by: wunhuang <wunhuang@amd.com>

feat: use py312 (sgl-project#8832)

4ef4783

fix: remove unused import (sgl-project#8809)

556e414

Add initial support for gpt-oss (sgl-project#8824)

c1d2061

chore: upgrade torch 2.8.0 (sgl-project#8836)

3ae8e3e

[router] complete router oai spec (sgl-project#8828)

5d62b56

Turn off hybrid cache by default (sgl-project#8839)

8128e08

Support bailing moe (sgl-project#8680)

d26ca84

[Feature] improve TBO: two chunk overlap (sgl-project#8144)

ca47e24

[router] PD Router Simplification and Reorganization (sgl-project#8838)

8c7bb39

[1/3] Optimize Slime Update Weights: Remove QWen3MOE Load Weight Over…

8958817

…head (sgl-project#8751)

[2/3] Optimize Slime Update Weights: Avoid GPU-to-CPU Device Sync whe…

cbbb738

…n update expert weights (sgl-project#8753)

tc-mb merged commit eff21cc into support-MiniCPM-V4.0 Aug 6, 2025
38 of 39 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sync main#1

sync main#1
tc-mb merged 396 commits intosupport-MiniCPM-V4.0from
main

tc-mb commented Aug 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

tc-mb commented Aug 6, 2025

Motivation

Modifications

Accuracy Test

Benchmark & Profiling

Checklist

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants