Skip to content

rebase to 2561ed0#10

Merged
ivanium merged 115 commits intomainfrom
pr-rebase-2561ed0
Sep 6, 2024
Merged

rebase to 2561ed0#10
ivanium merged 115 commits intomainfrom
pr-rebase-2561ed0

Conversation

@ZYHowell
Copy link
Copy Markdown
Collaborator

@ZYHowell ZYHowell commented Sep 2, 2024 •
edited
Loading

hnyls2002 and others added 30 commits August 15, 2024 00:31
@hnyls2002
Support stop_token_ids in sglang API (sgl-project#1092)
73cf683
@Ying1123
Support jinja as chat template file (sgl-project#1104)
6767e22
@merrymercy
Use a single workspace for flashinfer (sgl-project#1077)
326df4b
@Ying1123
[Fix] fix the typo bug for window attention (sgl-project#1106)
8d2d876
@merrymercy
Enable chunked prefill by default (sgl-project#1040)
e86b1cc
@Ying1123
[Fix] fix flashinfer usage for window attention (sgl-project#1107)
14cb544
@zhyncs
misc: rm unused model_loader (sgl-project#1110)
9195d13
@Ying1123
[Fix] Window attention compatible with RadixAttention and chunked pre…
93d4e35 
…fill (sgl-project#1112)
@merrymercy
set CUDA_DEVICE_MAX_CONNECTIONS=1 (sgl-project#1113)
0cb099e
@zhyncs
chore: bump v0.2.13 (sgl-project#1111)
5bd9537
@merrymercy
update hyperparameter guide (sgl-project#1114)
87a0db8
@zhyncs
ci: compatible with fork repo (sgl-project#1115)
26e9c12
@zhyncs
fix: resolve Python.h header missing (sgl-project#1119)
6aa8ad1
@merrymercy
Fix the deadlock in multi-node tp (sgl-project#1122)
5a261bd
@hnyls2002
Mixed style of chunked prefill (sgl-project#1013)
3694f8f
@hnyls2002
Fix port conflicts between local CI and runner CI. (sgl-project#1131)
f624f6a
@hnyls2002
Fix CI accuracy && time out limit (sgl-project#1133)
5d0d40d
@zhyncs
fix: use fp16 dtype for sm75 (sgl-project#1136)
9208591
@merrymercy
Improve the code style: more comments and remove useless packages (sg…
cdc8d60 
…l-project#1139)
@merrymercy
Improve benchmark (sgl-project#1140)
57d0bd9
@merrymercy
Fix duplicated imports in hf_transformers_utils.py (sgl-project#1141)
3c1f5a9
@min-xu-et
fixed a typo (sgl-project#1143)
fa13b95
@Michaelvll @concretevitamin
[Docs] Add instruction for running on clouds and kubernetes with SkyP…
d8627ed 
…ilot (sgl-project#1144)

Co-authored-by: Zongheng Yang <zongheng.y@gmail.com>
@yichuan-w @Ying1123
@zhyncs @merrymercy @hnyls2002
[Feat]Add support for optional start len of logprobs (sgl-project#1035)
b997a18 
Co-authored-by: Ying Sheng <sqy1415@gmail.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
Co-authored-by: Lianmin Zheng <lianminzheng@gmail.com>
Co-authored-by: Liangsheng Yin <hnyls2002@gmail.com>
@ispobock @zhyncs
Optimize MLA/GQA/MQA Triton decoding (sgl-project#1138)
df19125 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@vhain
feat: allow streaming for multi-prompt and/or parallel sampling (sgl-…
d847681 
…project#1134)
@merrymercy
Improve docs and warnings (sgl-project#1164)
a8ae640
@Xu-Chen @zhyncs
[Feature] add disable-custom-all-reduce (sgl-project#1148)
ff2cfdb 
Co-authored-by: chenxu02 <chenxu02@zhihu.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@zhyncs
misc: add hypervisor vendor (sgl-project#1165)
04707b0
@LucienShui @zhyncs
Generate 1 token to verify the health of the inference service in /he…
6242c39 
…alth (sgl-project#1154)

Co-authored-by: Yineng Zhang <me@zhyncs.com>
hnyls2002 and others added 29 commits August 28, 2024 18:58
@hnyls2002
Sampler cudagraph (sgl-project#1253)
381dd57
@lxww302
fix: multimodal_config in monkey_patch_vllm_dummy_weight_loader (sgl-…
a362340 
…project#1260)
@ispobock @zhyncs
Transpose mla weight offline (sgl-project#1261)
f414352 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@Deepfocused @zhyncs
EXAONE 3.0 Model Support (sgl-project#1258)
b7f8341 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@Deepfocused @zhyncs
Update README.md - Supported Models add Exaone 3.0 (sgl-project#1267)
55f5976 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@merrymercy
Report median instead of mean in bench_latency.py (sgl-project#1269)
79ece2c
@BabyChouSr
Allow more flexible assistant and system response (sgl-project#1256)
51c554d
@zhyncs
fix: resolve the fp8 bug introduced by vLLM 0.5.5 (sgl-project#1276)
52cefdb
@ByronHsu
[doc] fix quick start link (sgl-project#1282)
6cc9c52
@xiaobochen123
Optimize the update flashinfer indices (sgl-project#1262)
d134c13
@merrymercy
[CI] Add more multi-gpu tests (sgl-project#1280)
1b5d56f
@zhyncs @ispobock
feat: fix fp8 for MLA and support bmm fp8 for DeepSeek V2 (sgl-projec…
54772f7 
…t#1285)

Co-authored-by: ispobock <ispobaoke@163.com>
@merrymercy
[CI] merge all ci tests into one file (sgl-project#1289)
761b2ce
@ispobock @zhyncs
Support Triton fp8 e5m2 kv cache (sgl-project#1286)
6cb32ef 
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@ByronHsu
[triton] Remove the zero initialization of qk_acc by directly writing…
00b19f1 
… the result (sgl-project#1288)
@kevin85421 @zhyncs
[Chore] Rename model_overide_args to model_override_args (sgl-project…
0836055 
…#1284)

Signed-off-by: Kai-Hsun Chen <kaihsun@anyscale.com>
Co-authored-by: Yineng Zhang <me@zhyncs.com>
@qeternity
Allow new lines during JSON generation (sgl-project#1277)
32a4141
@zhyncs
fix: resolve fp8 for mixtral (sgl-project#1290)
9b08052
@zhyncs
ci: add nightly eval (sgl-project#1291)
6487ef6
@merrymercy
Fix the flaky tests in test_moe_eval_accuracy_large.py (sgl-project#1293
58fa607 
)
@ByronHsu
[doc] Fix more broken links (sgl-project#1294)
4a9f8ea
@hnyls2002
Fix regex mask (sgl-project#1296)
47f20da
@max99x
Fix hang when doing s += None. (sgl-project#1297)
6def9b0 
Co-authored-by: max99x <mshawabkeh@jamandtea.studio>
@merrymercy
Release v0.2.15 (sgl-project#1295)
9999442
@zhyncs
feat: update nightly gsm8k eval (sgl-project#1304)
2561ed0
@ZYHowell
Merge remote-tracking branch 'upstream/main' into pr-rebase-2561ed0
3b92e9d
@ZYHowell
minor fix
ef22804
@ZYHowell
contiguous and to device for triton kernel
daaef64
@ZYHowell
fix paged kernel lens before kv indptr
66ecf83
@ivanium ivanium merged commit 8b3662a into main Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.