[Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang by sufeng-buaa · Pull Request #12338 · sgl-project/sglang

sufeng-buaa · 2025-10-29T12:35:18Z

Motivation

The tracing package is implemented in sglang, so if mini_lb wants to enable tracing, it needs to import sglang.srt.tracing.trace. However, if the sgl-router image does not include sglang, this will cause an import error in mini_lb.

Thanks for reporting this bug @jinmingyi1998

Modifications

In mini_lb, check whether the tracing package is imported successfully; if not, disable tracing forcefully.

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

…ll sglang Signed-off-by: Feng Su <sufeng@linux.alibaba.com>

gemini-code-assist · 2025-10-29T12:35:35Z

Summary of Changes

Hello @sufeng-buaa, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a bug where the mini_lb component of the sgl-router would encounter an ImportError if tracing was enabled but the sglang package, which provides the necessary tracing utilities, was not present in the environment. The changes introduce robust error handling for tracing-related imports, ensuring that the system can gracefully disable tracing and continue operation without crashing, while also providing clear warnings to the user about the tracing status.

Highlights

Robust Tracing Imports: Implemented try-except ImportError blocks for sglang.srt.tracing.trace imports in both launch_router.py and mini_lb.py to prevent crashes when the sglang package is not installed.
Graceful Tracing Disablement: Tracing functionality is now automatically disabled if the sglang package, which provides the necessary tracing utilities, is not found in the environment.
Informative Warnings: A warning message is logged using logger.warning when tracing is not supported due to the absence of the sglang package, providing clear feedback to the user.
Conditional Tracing Calls: Calls to tracing functions like trace_req_start and trace_slice_start are now conditional on lb.enable_trace, ensuring they are only invoked when tracing is actively enabled and available.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request correctly fixes an ImportError when using mini_lb with tracing enabled in an environment where sglang is not installed. The fix involves adding try...except blocks to gracefully handle the missing dependency.

My review includes suggestions to refactor the implementation slightly. The goal is to improve encapsulation by moving all tracing-related logic for mini_lb into the MiniLoadBalancer class. This will make the code cleaner, remove redundancy, and make MiniLoadBalancer a more self-contained component. The proposed changes centralize the optional dependency check and initialization, improving maintainability.

gemini-code-assist · 2025-10-29T12:38:25Z

            if router_args.enable_trace:
-                from sglang.srt.tracing.trace import (
-                    process_tracing_init,
-                    trace_set_thread_info,
-                )
-
-                process_tracing_init(router_args.otlp_traces_endpoint, "sglang")
-                trace_set_thread_info("Mini lb")
+                try:
+                    from sglang.srt.tracing.trace import (
+                        process_tracing_init,
+                        trace_set_thread_info,
+                    )
+
+                    process_tracing_init(router_args.otlp_traces_endpoint, "sglang")
+                    trace_set_thread_info("Mini lb")
+                except ImportError:
+                    logger.warning(
+                        "Tracing is not supported in this environment. Please install sglang."
+                    )


To improve encapsulation, the tracing initialization logic could be moved into the MiniLoadBalancer class. This would centralize all tracing-related code and make launch_router.py simpler, as it would no longer need to handle tracing setup for mini_lb. I've added a suggestion in mini_lb.py that demonstrates how to implement this. With that change, this entire if router_args.enable_trace: block is no longer needed.

Should we set it to false when ImportError?

I update code according to gemini-code-assist and set enable_trace to false when import error.
take a look again?

gemini-code-assist · 2025-10-29T12:38:25Z

        self.prefill_bootstrap_ports = [url[1] for url in router_args.prefill_urls]
        self.decode_urls = router_args.decode_urls
-        self.enable_trace = router_args.enable_trace
+        self.enable_trace = trace_package_imported and router_args.enable_trace


To better encapsulate tracing logic, you could handle the check for trace_package_imported here and log a warning if tracing is requested but unavailable. This makes MiniLoadBalancer more self-contained.

As part of this refactoring, I'd also recommend moving the tracing initialization logic (process_tracing_init and trace_set_thread_info) from launch_router.py into the start() method of this class. This would require storing otlp_traces_endpoint from router_args in __init__, as shown in the suggestion.

Suggested change

self.enable_trace = trace_package_imported and router_args.enable_trace

self.otlp_traces_endpoint = router_args.otlp_traces_endpoint

self.enable_trace = router_args.enable_trace

if self.enable_trace and not trace_package_imported:

logger.warning(

"Tracing is not supported in this environment. Please install sglang."

)

self.enable_trace = False

Signed-off-by: Feng Su <sufeng@linux.alibaba.com>

sufeng-buaa · 2025-10-29T13:39:46Z

could you add a 'run_ci' tag please? @ShangmingCai

slin1237 · 2025-10-29T14:25:07Z

it shouldnt depend on sglang at all
circular dependency should be avoided at all time, for example
if you like to run mini lb anywhere thats ARM chip
this will fail because sglang has hard dependency on triton. and triton can not be installed on arm chip
but in theory, there is no reason why mini lb should not work on ARM chip
because its a super simple fastapi with proxy request to two servers
i think in the future, router code should block all changes that would like to add dependency from sglang engine
@key4ng
in router test module, we should get rid of sglang engine dependency
for example
unit test and integration tests(we currently use mock servers) should be runnable anywhere
only e2e should be using engines, and it should just use the cmd instead of the library

slin1237 · 2025-10-29T14:44:31Z

CI failure is not related to this change everything else LGTM

sufeng-buaa · 2025-10-30T01:48:05Z

it shouldnt depend on sglang at all circular dependency should be avoided at all time, for example if you like to run mini lb anywhere thats ARM chip this will fail because sglang has hard dependency on triton. and triton can not be installed on arm chip but in theory, there is no reason why mini lb should not work on ARM chip because its a super simple fastapi with proxy request to two servers i think in the future, router code should block all changes that would like to add dependency from sglang engine @key4ng in router test module, we should get rid of sglang engine dependency for example unit test and integration tests(we currently use mock servers) should be runnable anywhere only e2e should be using engines, and it should just use the cmd instead of the library

OK, I will implement a lightweight tracing package in mini_lb.

trace: fix import error in mini_lb if sgl-router image does not insta…

4d17830

…ll sglang Signed-off-by: Feng Su <sufeng@linux.alibaba.com>

sufeng-buaa requested review from CatherineSue, key4ng and slin1237 as code owners October 29, 2025 12:35

sufeng-buaa mentioned this pull request Oct 29, 2025

[Feature] Sglang Tracing: Fine-Grained Tracking for Request Latency - Part 2 #10804

Merged

4 tasks

gemini-code-assist Bot reviewed Oct 29, 2025

View reviewed changes

ShangmingCai assigned ShangmingCai and slin1237 Oct 29, 2025

trace: improve encapsulation according to code-assist

46cbbdd

Signed-off-by: Feng Su <sufeng@linux.alibaba.com>

ShangmingCai added the run-ci label Oct 29, 2025

slin1237 added bug Something isn't working high priority router labels Oct 29, 2025

slin1237 approved these changes Oct 29, 2025

View reviewed changes

slin1237 merged commit 1e90fe2 into sgl-project:main Oct 29, 2025
101 of 134 checks passed

sufeng-buaa mentioned this pull request Nov 10, 2025

Sglang Tracing: Decouple tracing from SGLang in mini_lb with lightweight implementation #12974

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang#12338

[Bug fix] trace: fix import error in mini_lb if sgl-router image does not install sglang#12338
slin1237 merged 2 commits intosgl-project:mainfrom
openanolis:sufeng-buaa/fix-trace-mini-lb

sufeng-buaa commented Oct 29, 2025

Uh oh!

gemini-code-assist Bot commented Oct 29, 2025

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot Oct 29, 2025

Uh oh!

ShangmingCai Oct 29, 2025

Uh oh!

sufeng-buaa Oct 29, 2025

Uh oh!

gemini-code-assist Bot Oct 29, 2025

Uh oh!

sufeng-buaa commented Oct 29, 2025

Uh oh!

slin1237 commented Oct 29, 2025 •

edited

Loading

Uh oh!

slin1237 commented Oct 29, 2025

Uh oh!

Uh oh!

sufeng-buaa commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

sufeng-buaa commented Oct 29, 2025

Motivation

Modifications

Checklist

Uh oh!

gemini-code-assist Bot commented Oct 29, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

ShangmingCai Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

sufeng-buaa Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

sufeng-buaa commented Oct 29, 2025

Uh oh!

slin1237 commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slin1237 commented Oct 29, 2025

Uh oh!

Uh oh!

sufeng-buaa commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

slin1237 commented Oct 29, 2025 •

edited

Loading