ci: unit test for `srt/observability` module by jiabinwa · Pull Request #21002 · sgl-project/sglang

jiabinwa · 2026-03-20T09:14:56Z

Motivation

Issue

#20865

Add unit tests for all modules under srt/observability/: label_transform, cpu_monitor, func_timer, req_time_stats, request_metrics_exporter, startup_func_log_and_timer, trace

Coverage

Module	Coverage
`label_transform.py`	100%
`cpu_monitor.py`	100%
`func_timer.py`	100%
`utils.py`	100%
`startup_func_log_and_timer.py`	100%
`req_time_stats.py`	98%
`request_metrics_exporter.py`	98%
`trace.py`	95%

Test

Short Version

.venv/bin/python -m pytest test/registered/unit/observability/ --cov --cov-config=.coveragerc --cov-report=term-missing 2>&1 | grep -E "(observability/|passed|failed|ERROR)" | tail -15

➜  sglang git:(jiabinwa-test/unit-tests-for-observability-module) .venv/bin/python -m pytest test/registered/unit/observability/ --cov --cov-config=.coveragerc --cov-report=term-missing 2>&1 | grep -E "(observability/|passed|failed|ERROR)" | tail -15
test/registered/unit/observability/test_label_transform.py ....          [  5%]
test/registered/unit/observability/test_metrics_utils.py ..........      [ 10%]
test/registered/unit/observability/test_req_time_stats.py .............. [ 17%]
test/registered/unit/observability/test_request_metrics_exporter.py .... [ 55%]
test/registered/unit/observability/test_startup_func_log_and_timer.py .. [ 65%]
test/registered/unit/observability/test_trace.py ....................... [ 82%]
python/sglang/srt/observability/cpu_monitor.py                                                          18      0   100%
python/sglang/srt/observability/func_timer.py                                                           42      0   100%
python/sglang/srt/observability/label_transform.py                                                      14      0   100%
python/sglang/srt/observability/req_time_stats.py                                                      521     12    98%   225, 925, 938-939, 954, 986-987, 1003, 1057-1060
python/sglang/srt/observability/request_metrics_exporter.py                                            107      2    98%   68, 188
python/sglang/srt/observability/startup_func_log_and_timer.py                                           60      0   100%
python/sglang/srt/observability/trace.py                                                               366     18    95%   61-67, 170-201
python/sglang/srt/observability/utils.py                                                                30      0   100%
======================= 197 passed, 2 warnings in 8.52s ========================

Full

sglang git:(jiabinwa-test/unit-tests-for-observability-module) ✗ .venv/bin/python -m pytest test/registered/unit/observability/
=========================================================================== test session starts ============================================================================
platform darwin -- Python 3.12.5, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/jiabwang/LinkedIn/sglang/sglang/test
configfile: pytest.ini
plugins: anyio-4.12.1, cov-7.0.0
collected 197 items

test/registered/unit/observability/test_cpu_monitor.py ..                                                                                                            [  1%]
test/registered/unit/observability/test_func_timer.py .....                                                                                                          [  3%]
test/registered/unit/observability/test_label_transform.py ....                                                                                                      [  5%]
test/registered/unit/observability/test_metrics_utils.py ..........                                                                                                  [ 10%]
test/registered/unit/observability/test_req_time_stats.py .....................................................................................                      [ 53%]
test/registered/unit/observability/test_request_metrics_exporter.py ......................                                                                           [ 64%]
test/registered/unit/observability/test_startup_func_log_and_timer.py ...........                                                                                    [ 70%]
test/registered/unit/observability/test_trace.py ..........................................................                                                          [100%]

============================================================================= warnings summary =============================================================================
.venv/lib/python3.12/site-packages/_pytest/config/__init__.py:1428
  /Users/jiabwang/LinkedIn/sglang/sglang/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py:1428: PytestConfigWarning: Unknown config option: asyncio_mode

    self._warn_or_fail_if_strict(f"Unknown config option: {key}\n")

.venv/lib/python3.12/site-packages/requests/__init__.py:113
  /Users/jiabwang/LinkedIn/sglang/sglang/.venv/lib/python3.12/site-packages/requests/__init__.py:113: RequestsDependencyWarning: urllib3 (2.6.3) or chardet (7.2.0)/charset_normalizer (3.4.6) doesn't match a supported version!
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===================================================================== 197 passed, 2 warnings in 1.48s ======================================================================

Two warning are irrelevant with this PR:

PytestConfigWarning: Unknown config option: asyncio_mode. The project's pytest.ini configures asyncio_mode for pytest-asyncio, which I didn't install in my local minimal venv. This warning doesn't appear in CI where the full test environment is set up.
RequestsDependencyWarning: urllib3 ... doesn't match a supported versio. A version mismatch between requests and urllib3 in our venv. Again, a local environment artifact.

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review Process

Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
After green CI and required approvals, ask Merge Oncalls to merge.

gemini-code-assist · 2026-03-20T09:15:26Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive set of unit tests for the srt/observability module. The primary goal is to ensure the correctness and robustness of the system's monitoring, timing, and tracing capabilities. By adding these tests, the changes aim to prevent regressions and provide a solid foundation for future enhancements to the observability stack, covering everything from CPU usage tracking to detailed request lifecycle tracing.

Highlights

Comprehensive Unit Test Coverage: Added a new suite of unit tests for various components within the srt/observability module, significantly enhancing test coverage and ensuring the reliability of monitoring and tracing functionalities.
Mocked CPU Monitor Tests: Introduced mocked tests for cpu_monitor.py to provide deterministic and fast validation of CPU usage delta calculations, ensuring accurate performance metrics.
New Test Suites for Observability Components: Included dedicated test files for func_timer, label_transform, req_time_stats, request_metrics_exporter, startup_func_log_and_timer, and trace modules, covering their respective functionalities in detail.
Validation of Metric Collection and Tracing: The new tests validate critical aspects such as metric enablement, time tracking, priority transformations, request lifecycle statistics, file-based metric exporting, and OpenTelemetry tracing mechanisms.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a comprehensive suite of unit tests for the srt/observability module, covering cpu_monitor, func_timer, label_transform, req_time_stats, startup_func_log_and_timer, and trace. The tests are well-structured, leveraging mocks and stubs effectively to ensure deterministic and lightweight execution. This is a great step towards improving the robustness and maintainability of the observability components. My feedback is minor and focuses on improving code style by centralizing imports.

Remove some comments

jiabinwa · 2026-03-20T09:48:35Z

Hi @ispobock @ping1jing2 @iforgetmyname @Edwardf0t1, thanks for reviewing! Happy to iterate on any feedback. Looking forward to contributing more to SGLang.

ispobock · 2026-03-20T16:10:40Z

/tag-and-rerun-ci

Pre-commit formatting & optional opentelemetry package

jiabinwa · 2026-03-21T03:03:08Z

PR Test / wait-for-stage-a (pull_request)

Hi @ispobock,

Thanks for helping run the CI. 8 jobs failed but only 2 are from this PR (both already fixed and pushed):

My fixes:

Task 1 (Lint): Pre-commit formatting (black/isort/ruff). Fixed.
Task 7 (stage-a-cpu-only): test_trace.py crashed due to opentelemetry not being installed in the CPU CI environment. Fixed by making the import conditional and skipping OTel-dependent tests when the package is unavailable. If the project wants full trace coverage in CI, opentelemetry could be added to the CPU test environment in the future.
Tasks 6 & 8: Cascading failures from Task 7.

Unrelated to this PR:

Task 3: test_completion_stream — IndexError on logprobs (existing E2E flake)
Task 4: test_multi_tokenizer_ttft — 89ms > 86ms threshold (AMD timing flake)
Task 5: test_tool_choice — Mistral-7B server failed to start, Connection refused (GPU infra issue)
Task 2: Rollup of Tasks 3–5

ispobock · 2026-03-21T16:02:30Z

+
+_ensure_module("sglang.srt.model_executor")
+_fbi = _ensure_module("sglang.srt.model_executor.forward_batch_info")
+_fbi.ForwardMode = _ForwardMode


how to handle the stub drift if the ForwardMode in forward_batch_info changed or refactored?

Thanks for the call out, this is a good point. The stub drift risk is real but bounded - if ForwardMode changes, req_time_stats.py behavior would change too, and the affected tests would need updating at that point, which would surface the stale stub.

That said, the stubs exist only because forward_batch_info.py transitively pulls in torch via sglang.srt.distributed, which isn't available in the CPU-only test environment. If ForwardMode (a simple IntEnum) were ever extracted into a lightweight module without heavy deps, we could drop the stub entirely and import the real class. Until then, I would suggest to keep the stub minimal to reduce the drift surface.

To avoid burden of communication, I added a single comment pointing to the source of truth. When ForwardMode changes, the developer updating req_time_stats.py will see the stub, see the comment, and know to update it. I think it is how stubs work with heavy deps.

jiabinwa changed the title ~~test: unit tests for srt/observability module~~ ci: [unit test] for srt/observability module Mar 20, 2026

gemini-code-assist Bot reviewed Mar 20, 2026

View reviewed changes

Comment thread test/registered/unit/observability/test_cpu_monitor.py Outdated

Comment thread test/registered/unit/observability/test_req_time_stats.py Outdated

Comment thread test/registered/unit/observability/test_trace.py Outdated

jiabinwa changed the title ~~ci: [unit test] for srt/observability module~~ ci: unit test for srt/observability module Mar 20, 2026

jiabinwa changed the title ~~ci: unit test for srt/observability module~~ ci: unit test for srt/observability module Mar 20, 2026

jiabinwa marked this pull request as ready for review March 20, 2026 09:30

test: unit tests for srt/observability module

2289671

jiabinwa force-pushed the jiabinwa-test/unit-tests-for-observability-module branch from 556b959 to c41c699 Compare March 20, 2026 09:35

Revise for Gemini Feedback

16fd19c

Remove some comments

jiabinwa force-pushed the jiabinwa-test/unit-tests-for-observability-module branch from c41c699 to 16fd19c Compare March 20, 2026 09:47

github-actions Bot added the run-ci label Mar 20, 2026

Revise

fb4b3ea

Pre-commit formatting & optional opentelemetry package

ispobock reviewed Mar 21, 2026

View reviewed changes

More comments

56aed7f

jiabinwa requested a review from ispobock March 22, 2026 20:39

ispobock merged commit 83b9d74 into sgl-project:main Mar 23, 2026
101 of 107 checks passed

adityavaid pushed a commit to adityavaid/sglang that referenced this pull request Mar 24, 2026

ci: unit test for srt/observability module (sgl-project#21002)

96b51dc

0-693 pushed a commit to 0-693/sglang that referenced this pull request Mar 25, 2026

ci: unit test for srt/observability module (sgl-project#21002)

500b628

JustinTong0323 pushed a commit to JustinTong0323/sglang that referenced this pull request Apr 7, 2026

ci: unit test for srt/observability module (sgl-project#21002)

c9b4d9a

Xio-Shark mentioned this pull request Apr 21, 2026

[Test] Add unit tests for observability label_transform and func_timer #21108

Open

5 tasks

yhyang201 pushed a commit to yhyang201/sglang that referenced this pull request Apr 22, 2026

ci: unit test for srt/observability module (sgl-project#21002)

45ea8f1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: unit test for `srt/observability` module#21002

ci: unit test for `srt/observability` module#21002
ispobock merged 4 commits intosgl-project:mainfrom
jiabinwa:jiabinwa-test/unit-tests-for-observability-module

jiabinwa commented Mar 20, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented Mar 20, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jiabinwa commented Mar 20, 2026

Uh oh!

ispobock commented Mar 20, 2026

Uh oh!

jiabinwa commented Mar 21, 2026

Uh oh!

ispobock Mar 21, 2026

Uh oh!

jiabinwa Mar 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jiabinwa commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Issue

Coverage

Test

Short Version

Full

Accuracy Tests

Benchmarking and Profiling

Checklist

Review Process

Uh oh!

gemini-code-assist Bot commented Mar 20, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jiabinwa commented Mar 20, 2026

Uh oh!

ispobock commented Mar 20, 2026

Uh oh!

jiabinwa commented Mar 21, 2026

Uh oh!

ispobock Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

jiabinwa Mar 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jiabinwa commented Mar 20, 2026 •

edited

Loading