Add option to return metadata in async streaming#18
Merged
Conversation
Ying1123
approved these changes
Jan 18, 2024
5 tasks
timethink
pushed a commit
to timethink/sglang
that referenced
this pull request
Mar 9, 2025
yanbing-j
added a commit
to yanbing-j/sglang
that referenced
this pull request
Mar 26, 2025
…project#18) * add rope * remove B * Fix issue * update * Add fused rope * refactor * add checks * support non-contiguous * update parallel
5 tasks
chunyuan-w
pushed a commit
to chunyuan-w/sglang
that referenced
this pull request
May 28, 2025
…project#18) * add rope * remove B * Fix issue * update * Add fused rope * refactor * add checks * support non-contiguous * update parallel
yanbing-j
added a commit
to yanbing-j/sglang
that referenced
this pull request
Jun 3, 2025
…project#18) * add rope * remove B * Fix issue * update * Add fused rope * refactor * add checks * support non-contiguous * update parallel
yanbing-j
added a commit
to yanbing-j/sglang
that referenced
this pull request
Jun 4, 2025
…project#18) * add rope * remove B * Fix issue * update * Add fused rope * refactor * add checks * support non-contiguous * update parallel
yanbing-j
added a commit
to yanbing-j/sglang
that referenced
this pull request
Jun 10, 2025
…project#18) * add rope * remove B * Fix issue * update * Add fused rope * refactor * add checks * support non-contiguous * update parallel
yanbing-j
added a commit
to yanbing-j/sglang
that referenced
this pull request
Jun 18, 2025
…project#18) * add rope * remove B * Fix issue * update * Add fused rope * refactor * add checks * support non-contiguous * update parallel
pengxin99
pushed a commit
to pengxin99/sglang
that referenced
this pull request
Jun 19, 2025
5 tasks
yichiche
pushed a commit
to yichiche/sglang
that referenced
this pull request
Jul 30, 2025
Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche
pushed a commit
to yichiche/sglang
that referenced
this pull request
Aug 7, 2025
Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
yichiche
pushed a commit
to yichiche/sglang
that referenced
this pull request
Aug 11, 2025
Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com>
5 tasks
5 tasks
zhuyijie88
pushed a commit
to zhuyijie88/sglang
that referenced
this pull request
Aug 28, 2025
Co-authored-by: xudaohong <xudaohong@huawei.com>
kalyank007
pushed a commit
to kalyank007/sglang
that referenced
this pull request
Nov 7, 2025
Co-authored-by: svc_repro_tool <svc_repro_tool@habana.ai>
5 tasks
nithinsubbiah
pushed a commit
to nithinsubbiah/sglang
that referenced
this pull request
Nov 21, 2025
Signed-off-by: Stanley Winata <stanley.winata@amd.com> [Wave] Add wave extend attention kernel Signed-off-by: Harsh Menon <harsh@nod-labs.com> [Wave] Adding logit_cap and layer scaling to API Also add support for the wave backend to the model runner. And use Triton decode kernels for now. [Wave] Run chunked prefill for perf comparison on Wave test Need to rename the non chunked/regular prefill version because otherwise rpd will treat it as the same kernel Signed-off-by: Stanley Winata <stanley.winata@amd.com> [Wave] Cache the function that loads the wave kernel Also maintain a global kernel hash to avoid recomputing the hash on every call. [Wave] Don't specify block size and enable buffer ops [Wave] Enable wave runtime and update scheduling API [Wave] Update API to use wave_compile & WaveCompileOptions [Wave] Update wave backend and extend attention to latest [Wave] Add speculative decode kernel Signed-off-by: nithinsubbiah <nithinsubbiah@gmail.com> cache kernels using lru_cache Update WaveBackend to use Wave Decode (sgl-project#6) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> Revert "Update WaveBackend to use Wave Decode (sgl-project#6)" (sgl-project#7) This reverts commit eac4599. Wave Backend decode (sgl-project#8) * align shapes Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> * fix Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> --------- Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> Wave backend fixes (sgl-project#10) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> More fixes to Wave decode (sgl-project#12) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> is_causal Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> Enable the grok in3 model (sgl-project#14) Set unique cache dir for each worker (sgl-project#16) update kernel (sgl-project#18) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> updated spec decode test as per wave Signed-off-by: xintin <gaurav.verma@amd.com> fix extend (sgl-project#23) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> Refactor paged decode intermediate arrays shapes (sgl-project#24) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> remove dyn symbols (sgl-project#26) Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> cleanup shapes (sgl-project#27) Some fields were removed from `paged_decode_attention_shape`. Signed-off-by: Ivan Butygin <ivan.butygin@gmail.com> Remove `mha` param from Wave decode attention kernel (sgl-project#28) Depends on iree-org/iree-turbine#1039 Signed-off-by: Paul Zhang <paul.zhang@amd.com> nfc: fix problems reported by linting update references from iree.turbine to wave_lang
Muqi1029
pushed a commit
to Muqi1029/sglang
that referenced
this pull request
Jan 4, 2026
…tr-usage-8ax32r Handle empty reasoning_tokens list in gRPC batch output
Garrybest
pushed a commit
to Garrybest/sglang
that referenced
this pull request
Jan 9, 2026
* fix moe class name
hnyls2002
pushed a commit
that referenced
this pull request
Feb 17, 2026
#18739) Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
5 tasks
magicYang1573
pushed a commit
to magicYang1573/sglang
that referenced
this pull request
Mar 9, 2026
…-project#18… (sgl-project#18739) Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
alphabetc1
pushed a commit
to alphabetc1/sglang
that referenced
this pull request
Mar 14, 2026
Add launcher to allow launching worker from router
Wangzheee
pushed a commit
to Wangzheee/sglang
that referenced
this pull request
Mar 21, 2026
…-project#18… (sgl-project#18739) Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: 赵晨阳 <zhaochen20@outlook.com>
mmangkad
pushed a commit
to mmangkad-dev/sglang
that referenced
this pull request
Apr 3, 2026
Tuning moe on h100.
Estrella-xx
added a commit
to Estrella-xx/sglang
that referenced
this pull request
Apr 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This allows the program to return both text and metadata for async streaming purposes.