Skip to content

[Diffusion] add rocm support#13492

Closed
sabreshao wants to merge 28 commits intosgl-project:mainfrom
sabreshao:diffusion_1118
Closed

[Diffusion] add rocm support#13492
sabreshao wants to merge 28 commits intosgl-project:mainfrom
sabreshao:diffusion_1118

Conversation

@sabreshao
Copy link
Copy Markdown
Contributor

@sabreshao sabreshao commented Nov 18, 2025 •
edited
Loading

Motivation

Enable sglang diffusion on AMD hardware.

Modifications

1, Add hip path for custom_op.
2, Use Aiter as default attention backend on ROCm.

Accuracy Tests

Verified on MI308x and H20 platform.
Setup on MI308x:

  • Docker: rocm/sgl-dev:v0.5.5.post3-rocm700-mi30x-20251117
  • pip uninstall sglang
  • python -m pip --no-cache-dir install -e "python[diffusion]"
  • pip install diffusers
  • pip install imageio-ffmpeg
  • pip install remote_pdb

Validation script:

from sglang.multimodal_gen import DiffGenerator
def main():
generator = DiffGenerator.from_pretrained(
model_path="Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=4,
)
prompt = "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest."
video = generator.generate(
prompt,
return_frames=True,
output_path="my_videos/",
save_output=True
)
if __name__ == '__main__':
main()

Benchmarking and Profiling

Checklist

@sabreshao sabreshao requested a review from mickqian as a code owner November 18, 2025 07:16
@gemini-code-assist
Copy link
Copy Markdown
Contributor

gemini-code-assist Bot commented Nov 18, 2025

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@github-actions github-actions Bot added the amd label Nov 18, 2025
@sabreshao sabreshao marked this pull request as draft November 18, 2025 07:16
@sabreshao sabreshao marked this pull request as ready for review November 18, 2025 16:09
@hubertlu-tw
Copy link
Copy Markdown
Collaborator

hubertlu-tw commented Nov 20, 2025

Hi @sabreshao,
Great work!
I am wondering if you could add multimodal-gen-test (https://github.com/sgl-project/sglang/blob/main/.github/workflows/pr-test.yml#L319-L343) to AMD CI (https://github.com/sgl-project/sglang/blob/main/.github/workflows/pr-test-amd.yml).

@hubertlu-tw
Copy link
Copy Markdown
Collaborator

hubertlu-tw commented Nov 20, 2025

/tag-and-rerun-ci

@sabreshao sabreshao mentioned this pull request Nov 20, 2025
Closed
@mickqian
Copy link
Copy Markdown
Collaborator

mickqian commented Nov 20, 2025

We have to add tests on amd before merging this

@mickqian mickqian changed the title [Diffusion] add rocm support. [Diffusion] add rocm support Nov 20, 2025
@zyzshishui
Copy link
Copy Markdown
Contributor

zyzshishui commented Nov 22, 2025

Hi @sabreshao, I’ve been working on a more detailed version of the AMD diffusion support. Do you mind if I file a new PR and add you as a co-author? Really appreciate your work!

@zyzshishui zyzshishui mentioned this pull request Nov 22, 2025
Merged
6 tasks
@github-actions github-actions Bot added dependencies Pull requests that update a dependency file diffusion SGLang Diffusion labels Dec 2, 2025
@sabreshao
Copy link
Copy Markdown
Contributor Author

sabreshao commented Dec 2, 2025

@hubertlu-tw #13743 merged into this PR.
@zyzshishui Thanks for contribution! I am trying to enable ROCm CI as soon as possible.

@ZiguanWang ZiguanWang mentioned this pull request Dec 17, 2025
Closed
6 tasks
@mickqian
Copy link
Copy Markdown
Collaborator

mickqian commented Jan 8, 2026

resolved in #13760

@mickqian mickqian closed this Jan 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mickqian mickqian Awaiting requested review from mickqian mickqian is a code owner

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner

@Ying1123 Ying1123 Awaiting requested review from Ying1123

@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner

@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner

@ch-wan ch-wan Awaiting requested review from ch-wan

@BBuf BBuf Awaiting requested review from BBuf BBuf is a code owner

@Edwardf0t1 Edwardf0t1 Awaiting requested review from Edwardf0t1

@yhyang201 yhyang201 Awaiting requested review from yhyang201 yhyang201 is a code owner

@ishandhanani ishandhanani Awaiting requested review from ishandhanani ishandhanani is a code owner

@Kangyan-Zhou Kangyan-Zhou Awaiting requested review from Kangyan-Zhou Kangyan-Zhou is a code owner

Assignees

No one assigned

Labels

amd dependencies Pull requests that update a dependency file diffusion SGLang Diffusion run-ci

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sabreshao @hubertlu-tw @mickqian @zyzshishui