Hicache sparse coordinator by xiezhq-hermann · Pull Request #16984 · sgl-project/sglang

xiezhq-hermann · 2026-01-13T00:46:22Z

Motivation

Open a PR for recording comments for iteration

# Startup
# prefill
nohup python3 -m sglang.launch_server --model-path /sgl-workspace/mnt/Qwen-8B --tp-size 1 --host 0.0.0.0 --port 10000 --page-size 64 --mem-fraction-static 0.8 --attention-backend fa3 --base-gpu-id 0  --disaggregation-mode prefill > sglang.out &

#decode
nohup python3 -m sglang.launch_server --model-path /sgl-workspace/mnt/Qwen-8B --tp-size 1 --host 0.0.0.0 --port 10010 --page-size 64 --mem-fraction-static 0.8 --attention-backend fa3 --base-gpu-id 2 --enable-hierarchical-sparse-attention --disable-overlap-schedule --disable-cuda-graph  --hierarchical-sparse-attention-extra-config='{"algorithm": "quest", "backend": "fa3", "fixed_topk_page_cnt":16}' --disaggregation-mode decode > sglang1.out &

#minilb
cd /sgl-workspace/sglang && export PYTHONPATH="/sgl-workspace/sglang/sgl-router/py_src:$PYTHONPATH"
nohup python -m sglang_router.launch_router --pd-disaggregation --mini-lb --prefill http://0.0.0.0:10000 --decode http://0.0.0.0:10010 --host 0.0.0.0 --port 20000 > lb.out &

# GSM8K Accuracy
nohup python benchmark/gsm8k/bench_sglang.py --port 20000 --num-questions 10 --num-shots 12 --parallel 1 > test.out&
100%|██████████| 100/100 [16:25<00:00,  9.86s/it]
Accuracy: 0.933
Invalid: 0.000

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review Process

Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
After green CI and required approvals, ask Merge Oncalls to merge.

Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>

gemini-code-assist · 2026-01-13T00:46:25Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

xiezhq-hermann · 2026-01-13T01:55:02Z

+    )
+
+
+def test_real_scenario():


can you provide a trace for benchmarking and performance tuning purpose? e.g., a collection of top k selection as input and a bechmark script measuring throughput.

…sparse_coordinator2 # Conflicts: # python/sglang/srt/mem_cache/common.py # python/sglang/srt/model_executor/model_runner.py

…host and device.

…ost and device. Co-authored-by: 1812107659@qq.com

hzh0425 and others added 2 commits January 11, 2026 19:58

Support first version

1f162ed

Support diff kernel

066b971

Co-authored-by: 晟海 <huangtingwei.htw@antgroup.com>

xiezhq-hermann requested review from BBuf, FlamingoPg, HaiShaw, Ying1123, ispobock, merrymercy, yizhang2077 and zhyncs as code owners January 13, 2026 00:46

xiezhq-hermann requested review from ByronHsu, Fridge003, Qiaolin-Yu, ShangmingCai, hanming-lu, hebiao064 and hnyls2002 as code owners January 13, 2026 00:46

github-actions Bot added the sgl-kernel label Jan 13, 2026

xiezhq-hermann assigned xiezhq-hermann, huangtingwei9988 and hzh0425 Jan 13, 2026