Skip to content

fix: adjusting vlm accuracy thresholds#16593

Merged
Kangyan-Zhou merged 2 commits intomainfrom
adjust-thresholds-vlm-accuracy
Jan 7, 2026
Merged

fix: adjusting vlm accuracy thresholds#16593
Kangyan-Zhou merged 2 commits intomainfrom
adjust-thresholds-vlm-accuracy

Conversation

@dougyster
Copy link
Copy Markdown
Collaborator

@dougyster dougyster commented Jan 6, 2026

Motivation

There are quite a few near misses that cause the test to fail; relaxing some thresholds to reduce test failures from near misses. Models adjusted:

  • zai-org/GLM-4.5V-FP8
  • moonshotai/Kimi-VL-A3B-Instruct
  • Qwen/Qwen2.5-VL-7B-Instruct
  • OpenGVLab/InternVL2_5-2B
  • unsloth/Mistral-Small-3.1-24B-Instruct-2503

re @merrymercy

reduce the threshold for all failed nightly-test-vlm-accuracy-2-gpu-runnerhttps://github.com/sgl-project/sglang/actions/runs/20513077000/job/58936524276#step:4:6851

Modifications

Increased latency threshold and lowered score threshold slightly.

Accuracy Tests

(https://github.com/sgl-project/sglang/actions/runs/20764464710/job/59627330523)

Benchmarking and Profiling

n/a

Checklist

Review Process

  1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments (/tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci) or contact authorized users to do so.
  4. After green CI and required approvals, ask Merge Oncalls to merge.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions Bot added the Multi-modal multi-modal language model label Jan 6, 2026
@Kangyan-Zhou Kangyan-Zhou merged commit 951d16c into main Jan 7, 2026
77 of 83 checks passed
@Kangyan-Zhou Kangyan-Zhou deleted the adjust-thresholds-vlm-accuracy branch January 7, 2026 03:41
michaelzhang-ai pushed a commit to michaelzhang-ai/sglang that referenced this pull request Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Multi-modal multi-modal language model

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants