Helios: Real Real-Time Long Video Generation Model by yhyang201 · Pull Request #19782 · sgl-project/sglang

yhyang201 · 2026-03-03T13:59:49Z

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.

Review Process

Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
Get approvals from CODEOWNERS and other reviewers.
Trigger CI tests with comments or contact authorized users to do so.
- /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
After green CI and required approvals, ask Merge Oncalls to merge.

- Restore fp8_utils.py and compressed_tensors_w4a4_mxint4_moe.py to main - Restore docs/diffusion/support_new_models.md to main - Remove Helios.md and add-new-diffusion-model.md skill

yhyang201 · 2026-03-03T14:00:08Z

/tag-and-rerun-ci

gemini-code-assist · 2026-03-03T14:00:25Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the SGLang framework by integrating the Helios video generation model family. It provides the foundational architecture for the Helios 3D Transformer, alongside comprehensive pipeline configurations tailored for different video generation stages, from basic Text-to-Video to advanced Pyramid Super-Resolution and Distilled models. A custom multi-stage scheduler and a sophisticated chunked denoising stage are introduced to efficiently handle the intricate requirements of high-quality video synthesis, including managing temporal dependencies and multi-resolution processing.

Highlights

New Model Integration: The Helios 3D Transformer model and its core configurations (HeliosConfig, HeliosArchConfig) have been integrated, enabling a new family of video generation capabilities.
Advanced Video Generation Pipelines: Specialized pipeline configurations (HeliosT2VConfig, HeliosMidConfig, HeliosDistilledConfig) are introduced for Text-to-Video, Pyramid Super-Resolution, and Distilled (DMD) video generation tasks.
Custom Multi-Stage Scheduler: A new HeliosScheduler has been implemented, supporting Euler, UniPC, and DMD solvers, featuring multi-stage sigma initialization and dynamic time shifting for enhanced denoising control.
Sophisticated Chunked Denoising Stage: The HeliosChunkedDenoisingStage is added to manage complex video generation processes, including multi-term memory history, CFG Zero Star guidance, and iterative denoising across video chunks and pyramid stages.
Dedicated Sampling Parameters: Specific sampling parameters (HeliosT2VSamplingParams, HeliosMidSamplingParams, HeliosDistilledSamplingParams) are defined for the various Helios models, including supported resolutions and negative prompts.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

python/sglang/multimodal_gen/configs/models/dits/init.py
- Integrated Helios model configuration.
python/sglang/multimodal_gen/configs/models/dits/helios.py
- Defined Helios 3D Transformer architecture configuration.
python/sglang/multimodal_gen/configs/pipeline_configs/init.py
- Integrated Helios pipeline configurations.
python/sglang/multimodal_gen/configs/pipeline_configs/helios.py
- Defined Helios T2V, Mid, and Distilled pipeline configurations.
python/sglang/multimodal_gen/configs/sample/helios.py
- Defined sampling parameters for Helios T2V, Mid, and Distilled models.
python/sglang/multimodal_gen/registry.py
- Registered Helios models and their configurations.
python/sglang/multimodal_gen/runtime/models/dits/helios.py
- Implemented the core Helios 3D Transformer model.
python/sglang/multimodal_gen/runtime/models/schedulers/scheduling_helios.py
- Implemented the Helios multi-stage scheduler.
python/sglang/multimodal_gen/runtime/pipelines/helios_pipeline.py
- Implemented the Helios video generation pipelines.
python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/helios_denoising.py
- Implemented the Helios chunked denoising stage.

Activity

The pull request is currently a 'work in progress' (wip) with an empty description, indicating it's in an early stage of development or awaiting further details.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request adds support for the Helios model family, including its base, mid, and distilled variants. It introduces new configuration files, a model implementation for the Helios 3D transformer, a custom scheduler, and a new pipeline. The changes are extensive and well-structured. I've identified a couple of areas for improvement related to an unused function parameter and an incorrect type hint, which should be straightforward to address.

python/sglang/multimodal_gen/configs/models/dits/helios.py

python/sglang/multimodal_gen/configs/pipeline_configs/helios.py

mickqian · 2026-03-04T03:42:14Z

python/sglang/multimodal_gen/registry.py

+        hf_model_paths=[
+            "BestWishYsh/Helios-Base",
+        ],
+        model_detectors=[


do we really need this? I'm considering deprecating model_detectors these days

yhyang201 · 2026-03-04T03:59:25Z

/rerun-failed-ci

yhyang201 · 2026-03-04T06:58:02Z

@mickqian all ci passed

yhyang201 added 7 commits March 1, 2026 03:24

upd

3aceb90

upd

831c25f

Merge origin/main into helios

d3a5e15

Revert unrelated changes from helios branch

c60e402

- Restore fp8_utils.py and compressed_tensors_w4a4_mxint4_moe.py to main - Restore docs/diffusion/support_new_models.md to main - Remove Helios.md and add-new-diffusion-model.md skill

upd

1ede6a9

upd

c04e2c1

upd

e4798d8

yhyang201 requested review from BBuf, mickqian, ping1jing2 and yingluosanqian as code owners March 3, 2026 13:59

github-actions bot added the diffusion SGLang Diffusion label Mar 3, 2026

github-actions bot added the run-ci label Mar 3, 2026

gemini-code-assist bot reviewed Mar 3, 2026

View reviewed changes

python/sglang/multimodal_gen/configs/models/dits/helios.py Show resolved Hide resolved

python/sglang/multimodal_gen/configs/pipeline_configs/helios.py Show resolved Hide resolved

mickqian reviewed Mar 4, 2026

View reviewed changes

yhyang201 changed the title ~~wip~~ Helios: Real Real-Time Long Video Generation Model Mar 4, 2026

yhyang201 added the auto-merge Experimental: automatically merge once approved and CI passes. label Mar 4, 2026

mickqian approved these changes Mar 4, 2026

View reviewed changes

yhyang201 merged commit 115f879 into sgl-project:main Mar 4, 2026
175 of 189 checks passed

Kangyan-Zhou pushed a commit to Kangyan-Zhou/sglang that referenced this pull request Mar 4, 2026

Helios: Real Real-Time Long Video Generation Model (sgl-project#19782)

3b8c1c2

qeternity pushed a commit to qeternity/sglang that referenced this pull request Mar 6, 2026

Helios: Real Real-Time Long Video Generation Model (sgl-project#19782)

c9dcd21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Helios: Real Real-Time Long Video Generation Model#19782

Helios: Real Real-Time Long Video Generation Model#19782
yhyang201 merged 7 commits intosgl-project:mainfrom
yhyang201:helios

yhyang201 commented Mar 3, 2026

Uh oh!

yhyang201 commented Mar 3, 2026

Uh oh!

gemini-code-assist bot commented Mar 3, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

mickqian Mar 4, 2026

Uh oh!

yhyang201 commented Mar 4, 2026

Uh oh!

yhyang201 commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

yhyang201 commented Mar 3, 2026

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

Review Process

Uh oh!

yhyang201 commented Mar 3, 2026

Uh oh!

gemini-code-assist bot commented Mar 3, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

mickqian Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

yhyang201 commented Mar 4, 2026

Uh oh!

yhyang201 commented Mar 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants