Skip to content

[New Model] GLM4.7-Flash#17247

Merged
ispobock merged 21 commits intomainfrom
temp
Jan 20, 2026
Merged

[New Model] GLM4.7-Flash#17247
ispobock merged 21 commits intomainfrom
temp

Conversation

@Qiaolin-Yu
Copy link
Copy Markdown
Collaborator

@Qiaolin-Yu Qiaolin-Yu commented Jan 17, 2026 •
edited by JustinTong0323
Loading

Motivation

Coauthored with Xinyuan

Modifications

Usage

Install sglang with this PR:

uv pip install sglang==0.3.2.dev9039+pr-17247.g90c446848 \                                                 
      --extra-index-url https://sgl-project.github.io/whl/pr/

Need to install transformers on this commit: huggingface/transformers@76732b4

uv pip install git+https://github.com/huggingface/transformers.git@76732b4e7120808ff989edbd16401f61fa6a0afa

For Blackweel, using --speculative-draft-attention-backend triton --attention-backend triton

Accuracy Tests

lm_eval \
  --model local-completions \
  --model_args model=zai-org/GLM-4.7-Flash,base_url=http://127.0.0.1:8000/v1/completions,tokenized_requests=False,trust_remote_code=True \
  --tasks gsm8k \
  --output_path ./ --batch_size 512
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8332|±  |0.0103|
|     |       |strict-match    |     5|exact_match|↑  |0.8340|±  |0.0102|

Benchmarking and Profiling

Checklist

Review Process

  1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
  4. After green CI and required approvals, ask Merge Oncalls to merge.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

gemini-code-assist Bot commented Jan 17, 2026

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@Qiaolin-Yu Qiaolin-Yu marked this pull request as draft January 17, 2026 06:06
@JustinTong0323 JustinTong0323 self-assigned this Jan 18, 2026
@JustinTong0323
lint
a74ef8f 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
Copy link
Copy Markdown
Collaborator

JustinTong0323 commented Jan 18, 2026 •
edited
Loading

Need to install transformers on this commit: huggingface/transformers@76732b4

uv pip install git+https://github.com/huggingface/transformers.git@76732b4e7120808ff989edbd16401f61fa6a0afa

For Blackweel, using --flashinfer-mla-disable-ragged

@JustinTong0323 JustinTong0323 changed the title wip: Temp [New Model] GLM4.7-Flash Jan 18, 2026
JustinTong0323 and others added 5 commits January 18, 2026 06:34
@JustinTong0323
transformers v5 fix: change return_dict=False in apply_caht_template
adf193f 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
remove unused attention implementation
2fbe129 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
Merge branch 'main' into temp
238d906
@JustinTong0323
remove dead config codes
90b9da4 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
Merge branch 'main' into temp
3030bf0
@JustinTong0323 JustinTong0323 marked this pull request as ready for review January 19, 2026 16:52
@JustinTong0323
Copy link
Copy Markdown
Collaborator

JustinTong0323 commented Jan 19, 2026 •
edited by b8zhong
Loading

/tag-and-rerun-ci again

Fridge003 and others added 2 commits January 20, 2026 01:10
@Fridge003
fix mha concat kernel
cec14de
@JustinTong0323
upd scaling
492a2ab 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
hardcode rope theta
d1f2688 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
Copy link
Copy Markdown
Collaborator

JustinTong0323 commented Jan 19, 2026

/tag-and-rerun-ci

@JustinTong0323
fix
0769e6b 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
fix mtp
72e94a1 
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@b8zhong b8zhong added the format Auto Format Code label Jan 19, 2026
@github-actions github-actions Bot removed the format Auto Format Code label Jan 19, 2026
@JustinTong0323
Copy link
Copy Markdown
Collaborator

JustinTong0323 commented Jan 20, 2026

/tag-and-rerun-ci

@ispobock ispobock merged commit 76b06be into main Jan 20, 2026
305 of 329 checks passed
@ispobock ispobock deleted the temp branch January 20, 2026 15:44
@montvid montvid mentioned this pull request Jan 29, 2026
Closed
2 tasks
Todobe pushed a commit to Todobe/sgl-sglang that referenced this pull request Mar 3, 2026
@Qiaolin-Yu @zRzRzRzRzRzRzR
@JustinTong0323 @Fridge003 @Todobe
[New Model] GLM4.7-Flash (sgl-project#17247)
3074f20 
Co-authored-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Co-authored-by: JustinTong0323 <justinning0323@gmail.com>
Co-authored-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
Co-authored-by: Baizhou Zhang <sobereddiezhang@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ispobock ispobock ispobock approved these changes

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner

@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner

@hebiao064 hebiao064 Awaiting requested review from hebiao064

@CatherineSue CatherineSue Awaiting requested review from CatherineSue CatherineSue is a code owner

@slin1237 slin1237 Awaiting requested review from slin1237 slin1237 is a code owner

@JustinTong0323 JustinTong0323 Awaiting requested review from JustinTong0323 JustinTong0323 is a code owner

Assignees

@JustinTong0323 JustinTong0323

Labels

high priority run-ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@Qiaolin-Yu @JustinTong0323 @ispobock @b8zhong @zRzRzRzRzRzRzR @Fridge003 @Kangyan-Zhou