Skip to content

[ROCm] enable moe topk softmax in amd#4448

Merged
merrymercy merged 1 commit intosgl-project:mainfrom
yiakwy-xpu-ml-framework-team:enable_moe_topk_softmax_in_amd
Mar 17, 2025
Merged

[ROCm] enable moe topk softmax in amd#4448
merrymercy merged 1 commit intosgl-project:mainfrom
yiakwy-xpu-ml-framework-team:enable_moe_topk_softmax_in_amd

Conversation

@yiakwy-xpu-ml-framework-team
Copy link
Copy Markdown
Contributor

@yiakwy-xpu-ml-framework-team yiakwy-xpu-ml-framework-team commented Mar 15, 2025

Motivation

Follow up of #4302

Modifications

  • bench test
截屏2025-03-15 13 24 28

Checklist

@yiakwy-xpu-ml-framework-team
Copy link
Copy Markdown
Contributor Author

yiakwy-xpu-ml-framework-team commented Mar 15, 2025

@HaiShaw

@yiakwy-xpu-ml-framework-team
Copy link
Copy Markdown
Contributor Author

Plan to enable it in SGLang critical path after merge, see #4439

Copy link
Copy Markdown
Collaborator

@BBuf BBuf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can merge this PR, as the only change is the export of amd topk_softmax interface. cc @zhyncs

@merrymercy merrymercy merged commit 9b8333d into sgl-project:main Mar 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants