Skip to content

Add NPU basic function testcases#19382

Merged
iforgetmyname merged 45 commits intosgl-project:mainfrom
shun8686:new-pr-commit
Mar 16, 2026
Merged

Add NPU basic function testcases#19382
iforgetmyname merged 45 commits intosgl-project:mainfrom
shun8686:new-pr-commit

Conversation

@Sugar920
Copy link
Copy Markdown
Contributor

@Sugar920 Sugar920 commented Feb 26, 2026

Motivation

This PR aims to comprehensively improve the test coverage of the Ascend (NPU) backend for the SGLang framework. A large number of targeted test cases have been added, and some existing test files have been optimized and adjusted. In total,48 files are involved (35 new files, 13 optimized files). The detailed changes are as follows:

1、Test Infrastructure Optimization
Adjusted the nightly-test-npu.yml workflow configuration, added the disaggregation_utils.py base class for PD disaggregation scenarios, and improved the test_ascend_utils.py test base class to support dependency management and model constant configuration for the new test cases.

2、Core Function Test Enhancement
Added dedicated test cases for basic functions including multi-level radix cache (L1/L2), expert parallelism, and speculative inference, covering validation for different input scenarios, models (DeepSeek, Qwen3, etc.), and performance metrics (TTFT).

3、Interface Layer Test Improvement
Fully supplemented Ascend platform API-related tests, covering scenarios such as chat/completions interfaces, function calling, stop matching, penalty, and profile performance analysis.

4、Model Test Iteration
Added or adjusted Ascend adaptation tests for various models including Gemma3, Phi4 multimodal, BGE reranker, and InternLM2 reward model; optimized existing test cases for models such as C4AI Command-R and DeepSeek V3.2. Slightly adjusted model test code for InternLM2, GLM4, etc., removed redundant logic, and improved the accuracy and integrity of test cases.

Modifications

1、Update nightly-test-npu.yml: Add test case dependency on third-party package installation.

2、Add disaggregation_utils.py:PD disaggregation scenario base class file.

3、Update the constant file test_ascend_utils.py, add model constants, and update model-related test cases.

4、Add model test cases, covering the models:
LLM Models, Reward Models, Rerank Models.

5、Add API test cases, covering the APIs:
/health; /health_generate; /ping; /model_info; /server_info; /get_load; /v1/models; /v1/models/{model:path}; /generate; /v1/chat/completions.

6、Add test cases for basic function:
L1 and L2 radix cache, deepep, speculative_inference.

Accuracy Tests

Benchmarking and Profiling

Checklist

Review Process

  1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
  4. After green CI and required approvals, ask Merge Oncalls to merge.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions Bot added Multi-modal multi-modal language model deepseek npu labels Feb 26, 2026
@iforgetmyname iforgetmyname self-assigned this Feb 26, 2026
@iforgetmyname
Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@classmethod
def setUpClass(cls):
cls.model = QWEN3_30B_A3B_WEIGHTS_PATH
cls.model = KIMI_K2_THINKING_WEIGHTS_PATH
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have qwen3-4b-thinking model & qwen3-30b-a3b-thinking model that shoud be suitable for this case

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a multimodal

Copy link
Copy Markdown
Collaborator

@iforgetmyname iforgetmyname left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try to avoid such a big pr later

@iforgetmyname iforgetmyname merged commit 895e560 into sgl-project:main Mar 16, 2026
86 of 94 checks passed
Wangzheee pushed a commit to Wangzheee/sglang that referenced this pull request Mar 21, 2026
Co-authored-by: cy <chenyang08056032@163.com>
Co-authored-by: Cherry_ming <136634645@qq.com>
0-693 pushed a commit to 0-693/sglang that referenced this pull request Mar 25, 2026
Co-authored-by: cy <chenyang08056032@163.com>
Co-authored-by: Cherry_ming <136634645@qq.com>
JustinTong0323 pushed a commit to JustinTong0323/sglang that referenced this pull request Apr 7, 2026
Co-authored-by: cy <chenyang08056032@163.com>
Co-authored-by: Cherry_ming <136634645@qq.com>
yhyang201 pushed a commit to yhyang201/sglang that referenced this pull request Apr 22, 2026
Co-authored-by: cy <chenyang08056032@163.com>
Co-authored-by: Cherry_ming <136634645@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek Multi-modal multi-modal language model npu run-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants