[Diffusion][MOVA] Fix task type in MOVA pipeline and shared model placement by CloudRipple · Pull Request #19489 · sgl-project/sglang

CloudRipple · 2026-02-27T08:00:43Z

Motivation

This pull request introduces several updates to the MOVA pipeline, focusing on device placement management for shared models and correcting the pipeline's task type. The changes improve resource handling when CPU offloading is enabled and clarify the model's intended use.

Modifications

Device placement and resource management improvements:

Added _ensure_shared_models_on_device method to mova.py to ensure that shared denoising modules (audio_dit and dual_tower_bridge) are placed on the correct device when CPU offload is enabled.
Updated the forward method in mova.py to use _ensure_shared_models_on_device instead of directly managing device placement for audio_dit.

Pipeline configuration correction:

Changed the task_type in MOVAPipelineConfig from ModelTaskType.T2V (text-to-video) to ModelTaskType.I2V (image-to-video), clarifying the pipeline's input/output modality.

Code clarity and maintenance:

Commented out the is_cross_attention=True parameter in the mova_dual_tower.py bridge model initialization.

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review Process

Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
After green CI and required approvals, ask Merge Oncalls to merge.

gemini-code-assist · 2026-02-27T08:01:01Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines the MOVA pipeline by improving resource management for shared models and correcting its operational definition. It ensures that critical denoising modules are properly placed on the device when CPU offloading is active, preventing potential issues. Additionally, it clarifies the pipeline's core function by accurately setting its task type, enhancing overall system correctness and maintainability.

Highlights

Enhanced Device Placement: Introduced a new method _ensure_shared_models_on_device to correctly manage the placement of shared denoising modules (audio_dit and dual_tower_bridge) on the active device, especially when CPU offloading is enabled.
Corrected Task Type: Updated the MOVAPipelineConfig to reflect the correct task_type as ModelTaskType.I2V (image-to-video) instead of ModelTaskType.T2V (text-to-video), clarifying the pipeline's intended input/output modality.
Code Clarity: Commented out the is_cross_attention=True parameter during the initialization of the bridge model in mova_dual_tower.py for improved code clarity.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

python/sglang/multimodal_gen/configs/pipeline_configs/mova.py
- Updated the task_type in MOVAPipelineConfig from ModelTaskType.T2V to ModelTaskType.I2V.
python/sglang/multimodal_gen/runtime/models/bridges/mova_dual_tower.py
- Commented out the is_cross_attention=True parameter in the __init__ method of the DualTowerBridge class.
python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/mova.py
- Added a new private method _ensure_shared_models_on_device to handle device placement for shared models.
- Modified the forward method to call _ensure_shared_models_on_device for managing shared model placement instead of direct _manage_device_placement calls for audio_dit.

Activity

The author has formatted the code according to pre-commit hooks.
The author has followed the SGLang code style guidance.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces several improvements to the MOVA pipeline. It correctly updates the task type to reflect its image-to-video nature, though a more precise type might be TI2V given it also processes text. The device placement logic for shared models during CPU offloading has been refactored and corrected, ensuring both audio_dit and dual_tower_bridge are on the correct device. I've also suggested a minor code cleanup to remove a commented-out parameter for better maintainability. Overall, these are good changes for correctness and robustness.

mickqian · 2026-02-27T11:51:14Z

            causal=False,
            softmax_scale=None,
-            is_cross_attention=True,
+            # is_cross_attention=True,


nit: why don't we remove it directly?

nit: why don't we remove it directly?

possibly enable in the future, or remove it in next refactor

mickqian · 2026-02-27T11:52:15Z

    """Configuration for MOVA (text+image -> video+audio) pipelines."""

-    task_type: ModelTaskType = ModelTaskType.T2V
+    task_type: ModelTaskType = ModelTaskType.I2V


does MOVA do T2V as well? in that case, we should opt for TI2V

does MOVA do T2V as well? in that case, we should opt for TI2V

It really is necessary to provide an image input now; text alone is not sufficient.

mickqian · 2026-02-27T11:55:51Z

/tag-and-rerun-ci

ping1jing2 · 2026-02-28T08:19:49Z

/rerun-failed-ci

…l placement (sgl-project#19489)

Fix task type in MOVA pipeline and shared model placement

349cd5e

CloudRipple requested review from mickqian, ping1jing2 and yhyang201 as code owners February 27, 2026 08:00

github-actions Bot added the diffusion SGLang Diffusion label Feb 27, 2026

gemini-code-assist Bot reviewed Feb 27, 2026

View reviewed changes

Comment thread python/sglang/multimodal_gen/configs/pipeline_configs/mova.py

Comment thread python/sglang/multimodal_gen/runtime/models/bridges/mova_dual_tower.py

RubiaCx added the run-ci label Feb 27, 2026

mickqian reviewed Feb 27, 2026

View reviewed changes

mickqian approved these changes Feb 27, 2026

View reviewed changes

fix: change default adjust_frames to false in MOVASamplingParams

f44a610

mickqian merged commit 4ec450e into sgl-project:main Mar 1, 2026
90 of 97 checks passed

tianyilt mentioned this pull request Mar 4, 2026

mova with sglang not working OpenMOSS/MOVA#39

Open

magicYang1573 pushed a commit to magicYang1573/sglang that referenced this pull request Mar 9, 2026

[diffusion][MOVA] fix: fix task type in MOVA pipeline and shared mode…

f6a945c

…l placement (sgl-project#19489)

Wangzheee pushed a commit to Wangzheee/sglang that referenced this pull request Mar 21, 2026

[diffusion][MOVA] fix: fix task type in MOVA pipeline and shared mode…

8f33051

…l placement (sgl-project#19489)

JustinTong0323 pushed a commit to JustinTong0323/sglang that referenced this pull request Apr 7, 2026

[diffusion][MOVA] fix: fix task type in MOVA pipeline and shared mode…

6bc54df

…l placement (sgl-project#19489)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Diffusion][MOVA] Fix task type in MOVA pipeline and shared model placement#19489

[Diffusion][MOVA] Fix task type in MOVA pipeline and shared model placement#19489
mickqian merged 2 commits intosgl-project:mainfrom
CloudRipple:mova/fix-bug

CloudRipple commented Feb 27, 2026

Uh oh!

gemini-code-assist Bot commented Feb 27, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

mickqian Feb 27, 2026

Uh oh!

CloudRipple Feb 27, 2026

Uh oh!

mickqian Feb 27, 2026

Uh oh!

CloudRipple Feb 27, 2026

Uh oh!

mickqian commented Feb 27, 2026

Uh oh!

ping1jing2 commented Feb 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

CloudRipple commented Feb 27, 2026

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Review Process

Uh oh!

gemini-code-assist Bot commented Feb 27, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

mickqian Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

CloudRipple Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

mickqian Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

CloudRipple Feb 27, 2026

Choose a reason for hiding this comment

Uh oh!

mickqian commented Feb 27, 2026

Uh oh!

ping1jing2 commented Feb 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants