feat(mm)(grpc): compute M-RoPE positions for preprocessed VL inputs#19973
Merged
feat(mm)(grpc): compute M-RoPE positions for preprocessed VL inputs#19973
Conversation
Collaborator
Author
|
/tag-run-ci-label |
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Collaborator
|
/tag-and-rerun-ci |
yhyang201
previously approved these changes
Mar 6, 2026
mickqian
reviewed
Mar 6, 2026
| if self.model_config.is_multimodal and self.processor is not None: | ||
| try: | ||
| from sglang.srt.managers.multimodal_processor import ( | ||
| get_mm_processor, |
Collaborator
There was a problem hiding this comment.
nit: move this to the top of this file?
Collaborator
Author
There was a problem hiding this comment.
Addressed. Sorry for the late reply.
| req.origin_input_ids, image_inputs | ||
| ) | ||
| req.extend_image_inputs(image_inputs) | ||
| self._maybe_compute_mrope_positions(req) |
Collaborator
There was a problem hiding this comment.
some duplication with: https://github.com/sgl-project/sglang/pull/19973/changes#diff-c3b8cc39d10c245933a25aa9c2fd6397f6b31ed8d85c0ecbb926c1f42afdd178L1844
we might need a refactor for this part later
Collaborator
There was a problem hiding this comment.
This link points to the initialization logic rather than the computation logic. Is it duplicated with this section?
Collaborator
Author
There was a problem hiding this comment.
I agree it is a bit duplicated. I couldn't think of a clear design due to the lack of familarity with the code here. Happy to refine this if you could point a direction.
9ad919d to
a419a9b
Compare
Collaborator
|
/rerun-failed-ci, |
Collaborator
|
/rerun-failed-ci |
The gRPC preprocessed path bypasses the model's multimodal processor, which normally computes M-RoPE positions during processing. Instead of adding model-specific M-RoPE logic to grpc_server.py (which defeats the purpose of a generic gRPC interface), this commit: - Adds compute_mrope_positions() to BaseMultimodalProcessor with a default no-op, overridden by QwenVL, GLM4V, and ERNIE4.5 processors - Loads the mm_processor in the scheduler (same pattern as vLLM) - Calls compute_mrope_positions() as a fallback when mrope_positions is None after multimodal input setup - Reverts all M-RoPE specific code from grpc_server.py, keeping it fully model-agnostic Signed-off-by: Chang Su <chang.s.su@oracle.com>
Signed-off-by: Chang Su <chang.s.su@oracle.com>
a419a9b to
e7030f7
Compare
Collaborator
|
/rerun-failed-ci |
Wangzheee
pushed a commit
to Wangzheee/sglang
that referenced
this pull request
Mar 21, 2026
…gl-project#19973) Signed-off-by: Chang Su <chang.s.su@oracle.com> Co-authored-by: Chang Su <chang.s.su@oracle.com>
0-693
pushed a commit
to 0-693/sglang
that referenced
this pull request
Mar 25, 2026
…gl-project#19973) Signed-off-by: Chang Su <chang.s.su@oracle.com> Co-authored-by: Chang Su <chang.s.su@oracle.com>
dutsc
pushed a commit
to dutsc/sglang
that referenced
this pull request
Mar 30, 2026
…gl-project#19973) Signed-off-by: Chang Su <chang.s.su@oracle.com> Co-authored-by: Chang Su <chang.s.su@oracle.com>
JustinTong0323
pushed a commit
to JustinTong0323/sglang
that referenced
this pull request
Apr 7, 2026
…gl-project#19973) Signed-off-by: Chang Su <chang.s.su@oracle.com> Co-authored-by: Chang Su <chang.s.su@oracle.com>
yhyang201
pushed a commit
to yhyang201/sglang
that referenced
this pull request
Apr 22, 2026
…gl-project#19973) Signed-off-by: Chang Su <chang.s.su@oracle.com> Co-authored-by: Chang Su <chang.s.su@oracle.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Qwen VL, GLM4V, and ERNIE4.5-VL models use Multi-dimensional Rotary Position Encoding (M-RoPE) which requires
mrope_positionscomputed fromimage_grid_thwand the expandedinput_ids. The normal HTTP path computes this inside the model's multimodal processor (e.g.,QwenVLProcessor), but the gRPC preprocessed path bypasses the processor entirely, leavingmrope_positionsasNoneand crashing in_compute_mrope_positions.Modifications
Rather than adding model-specific M-RoPE logic into
grpc_server.py(which would defeat the purpose of a generic gRPC interface), this PR:compute_mrope_positions()toBaseMultimodalProcessorwith a default no-op, overridden byQwenVLProcessor,GLM4VProcessor, andERNIE45VLProcessormm_processorin the scheduler (same pattern as vLLM)compute_mrope_positions()as a fallback whenmrope_positionsisNoneafter multimodal input setupgrpc_server.pyfully model-agnosticAccuracy Tests
N/A - This is a bug fix that enables gRPC preprocessed inputs to work with M-RoPE models. The M-RoPE position computation logic is identical to what the processors already compute in the HTTP path.
Benchmarking and Profiling
N/A - No performance-critical code path is modified. The M-RoPE position computation only runs when
mrope_positionsisNone(gRPC preprocessed path fallback).Checklist
Review Process
/tag-run-ci-label,/rerun-failed-ci,/tag-and-rerun-ci