Skip to content

Elementwise fused rmsnorm, fix triton error on ROCm#5124

Closed
HaiShaw wants to merge 1 commit intosgl-project:mainfrom
HaiShaw:rocm_fused_rmsnorm_fix
Closed

Elementwise fused rmsnorm, fix triton error on ROCm#5124
HaiShaw wants to merge 1 commit intosgl-project:mainfrom
HaiShaw:rocm_fused_rmsnorm_fix

Conversation

@HaiShaw
Copy link
Copy Markdown
Collaborator

@HaiShaw HaiShaw commented Apr 7, 2025 •
edited
Loading

Obsoleted by #5147

Motivation

Fix the triton error on ROCm.

Done by the co-authors:

Co-authored-by: David Hou

Modifications

As it is. Fix error below:

  File "/sgl-workspace/sglang/python/sglang/srt/layers/elementwise.py", line 260, in fused_rmsnorm
    fused_rmsnorm_kernel[(bs,)](
  File "/sgl-workspace/sglang/python/sglang/srt/models/grok.py", line 371, in forward
    fused_rmsnorm(
  File "/usr/local/lib/python3.12/dist-packages/triton/runtime/jit.py", line 330, in <lambda>
    return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
Traceback (most recent call last):
RuntimeError: Triton Error [HIP]:  Code: 1, Messsage: invalid argument

Checklist

@HaiShaw HaiShaw mentioned this pull request Apr 8, 2025
Merged
6 tasks
@zhyncs zhyncs closed this Apr 8, 2025
@HaiShaw HaiShaw deleted the rocm_fused_rmsnorm_fix branch April 8, 2025 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner

@Ying1123 Ying1123 Awaiting requested review from Ying1123 Ying1123 is a code owner

@zhyncs zhyncs Awaiting requested review from zhyncs

@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@HaiShaw @zhyncs