Skip to content

Support gpt oss moe lora#21375

Open
gongyisheng wants to merge 4 commits intosgl-project:mainfrom
gongyisheng:sglang-gpt-oss-moe-lora
Open

Support gpt oss moe lora#21375
gongyisheng wants to merge 4 commits intosgl-project:mainfrom
gongyisheng:sglang-gpt-oss-moe-lora

Conversation

@gongyisheng
Copy link
Copy Markdown
Contributor

@gongyisheng gongyisheng commented Mar 25, 2026 •
edited
Loading

Motivation

Support MoE LoRA for gpt-oss in miles

Modifications

This PR includes two parts

  • adapt to gpt-oss specific weight name
    • python/sglang/srt/server_args.py: use triton as moe backend when use gpt-oss model + lora adapter
    • python/sglang/srt/lora/mem_pool.py: support gpt-oss moe arch, which do not have shared experts
    • python/sglang/srt/lora/utils.py: gpt-oss moe layer name
    • python/sglang/srt/utils/hf_transformers_utils.py: gpt-oss specific configs
  • fix for load_lora
    • python/sglang/srt/managers/tp_worker.py: support loading lora in tp environment
    • python/sglang/srt/managers/io_struct.py: type hint change, since serialized_tensors is already a list[str]

todo:

  • add unittest

Accuracy Tests

N/A

Benchmarking and Profiling

N/A

Checklist

Review Process

  1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
  4. After green CI and required approvals, ask Merge Oncalls to merge.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

gemini-code-assist Bot commented Mar 25, 2026

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@gongyisheng gongyisheng mentioned this pull request Mar 25, 2026
Merged
@gongyisheng
Copy link
Copy Markdown
Contributor Author

gongyisheng commented Apr 1, 2026

@yushengsu-thu review request

@yushengsu-thu yushengsu-thu self-assigned this Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002 hnyls2002 is a code owner

@xiezhq-hermann xiezhq-hermann Awaiting requested review from xiezhq-hermann xiezhq-hermann is a code owner

@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner

@lifuhuang lifuhuang Awaiting requested review from lifuhuang lifuhuang is a code owner

@yushengsu-thu yushengsu-thu Awaiting requested review from yushengsu-thu yushengsu-thu is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@yushengsu-thu yushengsu-thu

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@gongyisheng @yushengsu-thu