Fix `torch.__version__` for PEP440 by EduardDurech · Pull Request #15682 · sgl-project/sglang

EduardDurech · 2025-12-23T12:05:06Z

https://peps.python.org/pep-0440

>>> "2.10"<"2.6"
True

gemini-code-assist · 2025-12-23T12:05:20Z

Summary of Changes

Hello @EduardDurech, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors how PyTorch version numbers are handled throughout the codebase. The primary goal is to eliminate potential bugs arising from incorrect string-based version comparisons, particularly when dealing with versions like '2.10' versus '2.6'. By adopting the packaging.version library, the system now performs accurate, PEP 440 compliant version checks, leading to more reliable and future-proof compatibility logic for different PyTorch releases.

Highlights

Robust Version Parsing: Introduced the packaging.version.parse utility to correctly interpret torch.__version__ strings, ensuring adherence to PEP 440 standards for version comparison.
Consistent Version Checks: Replaced direct string comparisons and startswith checks of torch.__version__ with comparisons against a pre-parsed torch_release object, preventing lexicographical comparison errors.
Improved PyTorch Compatibility Logic: Ensured that version-specific code paths, such as those related to torch._inductor and process group options, are correctly activated based on the actual PyTorch version.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request correctly addresses the issue of improper version string comparison by using packaging.version.parse as per PEP 440. The changes are sound. I've provided a couple of suggestions to improve maintainability by reducing code duplication and redundant imports. Overall, this is a good fix.

kartikx

Thank you for the PR, these changes are helpful.

The torch_release util can also help in other locations that are manually parsing the release.

For example:

sglang/python/sglang/srt/server_args.py

Lines 4562 to 4570 in 0d0367e

    
           parts = torch.__version__.split("+", 1)[0].split(".") 
        
           major = int(parts[0]) if len(parts) > 0 and parts[0].isdigit() else 0 
        
           minor = int(parts[1]) if len(parts) > 1 and parts[1].isdigit() else 0 
        
           if (major, minor) > (2, 6): 
        
               logger.warning( 
        
                   "WARNING: PD-Multiplexing may experience performance degradation with torch versions > 2.6.x.\n" 
        
                   f"  Current torch version is {torch.__version__}.\n" 
        
                   "  Please manually install torch 2.6.x." 
        
               )

sglang/python/sglang/srt/server_args.py

Lines 4627 to 4628 in 0d0367e

torch_version = torch.__version__.split("+", 1)[0]

if torch_version == "2.9.1":
sglang/python/sglang/srt/distributed/device_communicators/pynccl_allocator.py

Line 13 in 0d0367e

after_2_8_0 = version.parse(torch.__version__) >= version.parse("2.8.0")

EduardDurech · 2025-12-25T11:19:38Z

@kartikx updated all instances, if anyone has any suggestions please contribute review, I don't have time to maintain the PR (also if this could be merged? 😃)

kartikx · 2026-01-05T04:10:08Z

The changes look good to me, however I do not have CI perms.

@fzyzcjy Can you please take a look since you have also been doing refactoring changes recently?

kartikx · 2026-01-05T04:15:27Z

/gemini review

gemini-code-assist · 2026-01-05T04:15:30Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

Kangyan-Zhou · 2026-01-05T17:47:13Z

/tag-and-rerun-ci

EduardDurech · 2026-01-29T17:13:14Z

@Kangyan-Zhou @kartikx could one of you guys rerun the CLI and bring this up to admins? It's crazy such a basic, fundamental, standard PR takes a month to merge...

EduardDurech · 2026-01-29T18:58:38Z

The CI is broken, I'm no longer maintaining this PR, if SGLang doesn't merge it SGLang's problem

Kangyan-Zhou · 2026-01-29T19:56:04Z

@Kangyan-Zhou @kartikx could one of you guys rerun the CLI and bring this up to admins? It's crazy such a basic, fundamental, standard PR takes a month to merge...

Yes we also realized our review process is completely broken, and are working to improve it asap. Sry for the long review and merge process.

EduardDurech · 2026-01-30T00:09:16Z

@Kangyan-Zhou thank you, I know you guys have a lot and it's difficult to manage 1.5k PRs but it blocks us quite often for doing development, is it possible to have a fast-track or some internal chat with you guys when we want to do PRs? We're doing things at scale (>10k GH200) and fully transparent with data, models, repos, we have our model PR'd to SGLang and are constantly training new versions https://huggingface.co/swiss-ai, it would be nice if we could have a bit more contact with you guys because I'm waiting on average 1 month for PRs now and have to rebase

We're quite active with SGLang and multiple people, we'd like to contribute but the current workflow is very unappealing for contributors, some example PRs

#9156
https://github.com/sgl-project/sglang/issues?q=is%3Apr%20author%3Anathanrchn
https://github.com/sgl-project/sglang/issues?q=is%3Apr%20author%3AEduardDurech

Thank you

Kangyan-Zhou · 2026-01-30T00:17:50Z

@EduardDurech could you please join sglang slack and dm me? Thanks!

mickqian · 2026-01-30T03:02:49Z

    get_sp_group,
    get_ulysses_parallel_world_size,
 )
+from sglang.srt.utils.common import torch_release


please avoid importing sglang.srt in multimodal_gen

@mickqian would you like me to make a version utils? Or just duplicate the torch vesrion in https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/utils.py?

EduardDurech requested a review from hebiao064 as a code owner December 23, 2025 12:05

github-actions Bot added the piecewise-cuda-graph label Dec 23, 2025

This was referenced Dec 23, 2025

Online weight updates from torch.distributed #2279

Merged

Piecewise CUDA Graph Support & Torch Compile Backend #10062

Merged

gemini-code-assist Bot reviewed Dec 23, 2025

View reviewed changes

Comment thread python/sglang/srt/compilation/compiler_interface.py Outdated

Comment thread python/sglang/srt/utils/common.py Outdated

EduardDurech force-pushed the fix/torch__version__ branch from 076938c to 81fbd97 Compare December 23, 2025 12:22

kartikx reviewed Dec 23, 2025

View reviewed changes

EduardDurech requested review from AniZpZ, BBuf, Edwardf0t1, FlamingoPg, ch-wan, merrymercy, mickqian, yhyang201 and yizhang2077 as code owners December 25, 2025 11:18

github-actions Bot added the diffusion SGLang Diffusion label Dec 25, 2025

mickqian removed the diffusion SGLang Diffusion label Dec 30, 2025

adarshxs added the express-lane label Jan 4, 2026

kartikx reviewed Jan 5, 2026

View reviewed changes

Comment thread python/sglang/srt/server_args.py Outdated

github-actions Bot added run-ci diffusion SGLang Diffusion labels Jan 5, 2026

kartikx approved these changes Jan 6, 2026

View reviewed changes

EduardDurech added 3 commits January 29, 2026 18:16

Fix torch.__version__ for PEP440

c1d5ae7

Utils for all instances

97d116a

Indexed

07d6fec

EduardDurech force-pushed the fix/torch__version__ branch from 7b0be5b to 07d6fec Compare January 29, 2026 17:34

Kangyan-Zhou merged commit 1b6798a into sgl-project:main Jan 29, 2026
73 of 94 checks passed

mickqian reviewed Jan 30, 2026

View reviewed changes

charlesHsuGG pushed a commit to charlesHsuGG/sglang that referenced this pull request Jan 30, 2026

Fix torch.__version__ for PEP440 (sgl-project#15682)

83c0c0a

Chen-0210 pushed a commit to Chen-0210/sglang that referenced this pull request Jan 30, 2026

Fix torch.__version__ for PEP440 (sgl-project#15682)

368ba3a

sfiisf pushed a commit to sfiisf/sglang that referenced this pull request Feb 5, 2026

Fix torch.__version__ for PEP440 (sgl-project#15682)

616198a

Johnsonms pushed a commit to Johnsonms/sglang that referenced this pull request Feb 14, 2026

Fix torch.__version__ for PEP440 (sgl-project#15682)

34e56cc

	parts = torch.__version__.split("+", 1)[0].split(".")
	major = int(parts[0]) if len(parts) > 0 and parts[0].isdigit() else 0
	minor = int(parts[1]) if len(parts) > 1 and parts[1].isdigit() else 0
	if (major, minor) > (2, 6):
	logger.warning(
	"WARNING: PD-Multiplexing may experience performance degradation with torch versions > 2.6.x.\n"
	f" Current torch version is {torch.__version__}.\n"
	" Please manually install torch 2.6.x."
	)

	torch_version = torch.__version__.split("+", 1)[0]
	if torch_version == "2.9.1":

Conversation

EduardDurech commented Dec 23, 2025

Uh oh!

gemini-code-assist Bot commented Dec 23, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

kartikx left a comment

Choose a reason for hiding this comment

Uh oh!

EduardDurech commented Dec 25, 2025

Uh oh!

kartikx commented Jan 5, 2026

Uh oh!

Uh oh!

kartikx commented Jan 5, 2026

Uh oh!

gemini-code-assist Bot commented Jan 5, 2026

Uh oh!

Kangyan-Zhou commented Jan 5, 2026

Uh oh!

EduardDurech commented Jan 29, 2026

Uh oh!

EduardDurech commented Jan 29, 2026

Uh oh!

Uh oh!

Kangyan-Zhou commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

EduardDurech commented Jan 30, 2026

Uh oh!

Kangyan-Zhou commented Jan 30, 2026

Uh oh!

mickqian Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

EduardDurech Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kangyan-Zhou commented Jan 29, 2026 •

edited

Loading

mickqian Jan 30, 2026 •

edited

Loading

EduardDurech Jan 30, 2026 •

edited

Loading