Skip to content

[Fix] suppress remote weight loading engine w/o mooncake installed#14937

Merged
Kangyan-Zhou merged 5 commits intosgl-project:mainfrom
ZailiWang:fix-mooncake-dep
Dec 12, 2025
Merged

[Fix] suppress remote weight loading engine w/o mooncake installed#14937
Kangyan-Zhou merged 5 commits intosgl-project:mainfrom
ZailiWang:fix-mooncake-dep

Conversation

@ZailiWang
Copy link
Copy Markdown
Contributor

@ZailiWang ZailiWang commented Dec 12, 2025 •
edited
Loading

Motivation

Fix the ModuleNotFoundError: No module named 'mooncake' in non-cuda CI cases introduced by #13125

Modifications

Add mooncake package check with try...except syntax. Set server_args.remote_instance_weight_loader_support_transfer_engine as False if mooncake package was not installed.

Adding mooncake installation does not work on non-CUDA platforms. Such error would be raised:

  File "/opt/.venv/lib/python3.12/site-packages/sglang/srt/managers/scheduler.py", line 2683, in run_scheduler_process
    scheduler = Scheduler(
                ^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/sglang/srt/managers/scheduler.py", line 320, in __init__
    self.tp_worker = TpModelWorker(
                     ^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/sglang/srt/managers/tp_worker.py", line 248, in __init__
    self._model_runner = ModelRunner(
                         ^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/sglang/srt/model_executor/model_runner.py", line 366, in __init__
    self.initialize(min_per_gpu_memory)
  File "/opt/.venv/lib/python3.12/site-packages/sglang/srt/model_executor/model_runner.py", line 451, in initialize
    register_memory_region_v2(
  File "/opt/.venv/lib/python3.12/site-packages/sglang/srt/model_loader/remote_instance_weight_loader_utils.py", line 144, in register_memory_region_v2
    memory_snapshot = torch.cuda.memory.memory_snapshot()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib/python3.12/site-packages/torch/cuda/memory.py", line 627, in memory_snapshot
    return torch._C._cuda_memorySnapshot(mempool_id)["segments"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'torch._C' has no attribute '_cuda_memorySnapshot'

@gemini-code-assist
Copy link
Copy Markdown
Contributor

gemini-code-assist Bot commented Dec 12, 2025

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@ZailiWang
Copy link
Copy Markdown
Contributor Author

ZailiWang commented Dec 12, 2025

/tag-run-ci-label

@ZailiWang
Copy link
Copy Markdown
Contributor Author

ZailiWang commented Dec 12, 2025

The CI failure on Xeon was observed for #13125: https://github.com/sgl-project/sglang/actions/runs/20143344123/job/57816878503?pr=13125 but somehow merged regardlessly :(

@mingfeima
Copy link
Copy Markdown
Collaborator

mingfeima commented Dec 12, 2025

@MingxuZh please check CI status.

@ZailiWang
Copy link
Copy Markdown
Contributor Author

ZailiWang commented Dec 12, 2025

Xeon CI passed: https://github.com/sgl-project/sglang/actions/runs/20155160665/job/57855963553?pr=14937

@Kangyan-Zhou Kangyan-Zhou merged commit 4dabfbc into sgl-project:main Dec 12, 2025
56 of 88 checks passed
zhyncs added a commit that referenced this pull request Dec 12, 2025
@zhyncs
Revert "[Fix] suppress remote weight loading engine w/o mooncake inst…
eb4d3d5 
…alled (#14937)"

This reverts commit 4dabfbc.
@ZailiWang ZailiWang deleted the fix-mooncake-dep branch December 12, 2025 07:27
@ZailiWang ZailiWang mentioned this pull request Dec 12, 2025
Merged
6 tasks
BenYao21 pushed a commit to minleminzui/sglang that referenced this pull request Dec 13, 2025
@ZailiWang @BenYao21
[Fix] suppress remote weight loading engine w/o mooncake installed (s…
5d6dbfd 
…gl-project#14937)
Prozac614 pushed a commit to Prozac614/sglang that referenced this pull request Dec 17, 2025
@ZailiWang @Prozac614
[Fix] suppress remote weight loading engine w/o mooncake installed (s…
7f5a98c 
…gl-project#14937)
YChange01 pushed a commit to YChange01/sglang that referenced this pull request Jan 13, 2026
@ZailiWang @YChange01
[Fix] suppress remote weight loading engine w/o mooncake installed (s…
b587dd1 
…gl-project#14937)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

high priority run-ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ZailiWang @mingfeima @Kangyan-Zhou