Skip to content

[Bug] Qwen3 MoE fails to load when config uses rope_theta without rope_parameters #20932

@lviy

Description

@lviy

Checklist

  • I searched related issues but found no solution.
  • The bug persists in the latest version.
  • Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback.
  • If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
  • Please use English. Otherwise, it will be closed.

Describe the bug

SGLang fails to load some Qwen3 MoE checkpoints because qwen3_moe.py assumes config.rope_parameters["rope_theta"] always exists.
However, some valid Qwen3 configs expose RoPE as top-level rope_theta (and optional rope_scaling) without rope_parameters, which
triggers an AttributeError.

Expected behavior is to support both config styles:

  • new style: config.rope_parameters["rope_theta"]
  • old style: config.rope_theta / config.rope_scaling

A compatibility fallback in qwen3_moe.py would fix this and avoid startup failures across different Qwen3 checkpoint formats.

I will submit a PR to address this issue. #20931
Image

Reproduction

python3 -m sglang.launch_server
--model-path /gfs/space/chatrl/public/models/Qwen3-30B-A3B-Instruct-2507
--trust-remote-code
--host 0.0.0.0
--port 30000
--ep-size 2
--tp-size 2 \

Environment

root@d68dddca53dc1ca316399de49c20639c-taskrole1-0:/gfs/platform/public/infra/lxr/sglang# python3 -m sglang.check_env
Python: 3.12.3 (main, Nov 6 2025, 13:44:16) [GCC 13.3.0]
CUDA available: True
GPU 0,1: NVIDIA H800 80GB
GPU 0,1 Compute Capability: 9.0
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 12.9, V12.9.86
CUDA Driver Version: 580.105.08
PyTorch: 2.9.1+cu129
sglang: 0.0.0.dev1+gdce8b0606
sgl_kernel: 0.3.21
flashinfer_python: 0.5.3
flashinfer_cubin: 0.5.3
flashinfer_jit_cache: 0.5.3+cu129
triton: 3.5.1
transformers: 4.57.1
torchao: 0.9.0
numpy: 1.26.4
aiohttp: 3.13.3
fastapi: 0.128.0
hf_transfer: 0.1.9
huggingface_hub: 0.36.0
interegular: 0.3.3
modelscope: 1.33.0
orjson: 3.11.5
outlines: 0.1.11
packaging: 25.0
psutil: 7.2.1
pydantic: 2.12.5
python-multipart: 0.0.21
pyzmq: 27.1.0
uvicorn: 0.40.0
uvloop: 0.22.1
vllm: Module Not Found
xgrammar: 0.1.27
openai: 2.6.1
tiktoken: 0.12.0
anthropic: 0.75.0
litellm: Module Not Found
decord2: 3.0.0
NVIDIA Topology:
GPU0 GPU1 NIC0 NIC1 NIC2 NIC3 NIC4 NIC5 NIC6 NIC7 NIC8 NIC9 CPU Affinity NUMA Affinity GPU NUMA ID
GPU0 X NV18 SYS PXB PXB PIX PXB SYS SYS SYS SYS PXB 0-45,92-137 0 N/A
GPU1 NV18 X SYS PXB PXB PXB PIX SYS SYS SYS SYS PXB 0-45,92-137 0 N/A
NIC0 SYS SYS X SYS SYS SYS SYS SYS SYS SYS SYS SYS
NIC1 PXB PXB SYS X PXB PXB PXB SYS SYS SYS SYS PXB
NIC2 PXB PXB SYS PXB X PXB PXB SYS SYS SYS SYS PXB
NIC3 PIX PXB SYS PXB PXB X PXB SYS SYS SYS SYS PXB
NIC4 PXB PIX SYS PXB PXB PXB X SYS SYS SYS SYS PXB
NIC5 SYS SYS SYS SYS SYS SYS SYS X PXB PXB PXB SYS
NIC6 SYS SYS SYS SYS SYS SYS SYS PXB X PXB PXB SYS
NIC7 SYS SYS SYS SYS SYS SYS SYS PXB PXB X PXB SYS
NIC8 SYS SYS SYS SYS SYS SYS SYS PXB PXB PXB X SYS
NIC9 PXB PXB SYS PXB PXB PXB PXB SYS SYS SYS SYS X

Legend:

X = Self
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX = Connection traversing at most a single PCIe bridge
NV# = Connection traversing a bonded set of # NVLinks

NIC Legend:

NIC0: mlx5_0
NIC1: mlx5_gdr_0
NIC2: mlx5_gdr_1
NIC3: mlx5_gdr_2
NIC4: mlx5_gdr_3
NIC5: mlx5_gdr_4
NIC6: mlx5_gdr_5
NIC7: mlx5_gdr_6
NIC8: mlx5_gdr_7
NIC9: mlx5_stor_0

ulimit soft: 1048576

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions