Skip to content

[ROCm][TunableOp] Fix ScaledGEMM rowwise#152403

Closed
naromero77amd wants to merge 3 commits intopytorch:mainfrom
ROCm:fix_rocm_scaled_gemm_rowwise
Closed

[ROCm][TunableOp] Fix ScaledGEMM rowwise#152403
naromero77amd wants to merge 3 commits intopytorch:mainfrom
ROCm:fix_rocm_scaled_gemm_rowwise

Conversation

@naromero77amd
Copy link
Collaborator

@naromero77amd naromero77amd commented Apr 29, 2025 •
edited
Loading

Fixes TunableOp ScaledGEMM regression for rowwise scaling caused by this #147548

Credit goes to @mawong-amd for fix.

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang

@pytorch-bot
Copy link

pytorch-bot bot commented Apr 29, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/152403

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit b7880c2 with merge base e7c19f4 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added module: rocm AMD GPU support for Pytorch ciflow/rocm Trigger "default" config CI on ROCm labels Apr 29, 2025
@naromero77amd naromero77amd removed the ciflow/rocm Trigger "default" config CI on ROCm label Apr 29, 2025
@naromero77amd naromero77amd marked this pull request as draft April 29, 2025 04:17
@naromero77amd naromero77amd requested a review from jeffdaily April 29, 2025 04:17
@naromero77amd naromero77amd added topic: not user facing topic category ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 labels Apr 29, 2025
@naromero77amd naromero77amd marked this pull request as ready for review April 29, 2025 04:21
@naromero77amd naromero77amd force-pushed the fix_rocm_scaled_gemm_rowwise branch from c7afaf8 to 21a324c Compare April 29, 2025 17:54
@pytorch-bot pytorch-bot bot removed the ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 label Apr 29, 2025
@naromero77amd naromero77amd changed the title [ROCm] Fix ScaledGEMM rowwise descriptor. [ROCm][TunableOp] Fix ScaledGEMM rowwise Apr 29, 2025
@pytorch-bot pytorch-bot bot added the ciflow/rocm Trigger "default" config CI on ROCm label Apr 29, 2025
@naromero77amd naromero77amd added ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 and removed ciflow/rocm Trigger "default" config CI on ROCm labels Apr 29, 2025
@pytorch-bot pytorch-bot bot removed the ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 label Apr 29, 2025
@jeffdaily jeffdaily added ciflow/rocm Trigger "default" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 labels Apr 29, 2025
@naromero77amd
Copy link
Collaborator Author

naromero77amd commented Apr 30, 2025

Confirming that new UT completed successfully in test(default 3, 6, linux.rocm.gpu.mi300.2):

test/test_linalg.py::TestLinalgCUDA::test_rotating_buffer_tunableop_cuda_float32, test/test_linalg.py::TestLinalgCUDA::test_rowwise_scaled_gemm_numerics_tunableop_cuda_float8_e4m3fnuz,

@naromero77amd
Copy link
Collaborator Author

naromero77amd commented Apr 30, 2025

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Apr 30, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Apr 30, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@naromero77amd naromero77amd deleted the fix_rocm_scaled_gemm_rowwise branch April 30, 2025 17:26
naromero77amd added a commit to ROCm/pytorch that referenced this pull request May 8, 2025
@naromero77amd
[ROCm][TunableOp] Fix ScaledGEMM rowwise (pytorch#152403)
63bd33c 
Fixes TunableOp ScaledGEMM regression for rowwise scaling caused by this pytorch#147548

Credit goes to @mawong-amd for fix.

Pull Request resolved: pytorch#152403
Approved by: https://github.com/jeffdaily

(cherry picked from commit ece1658)
@naromero77amd naromero77amd mentioned this pull request May 8, 2025
Merged
pruthvistony pushed a commit to ROCm/pytorch that referenced this pull request May 8, 2025
@naromero77amd @jeffdaily
[release/2.7][ROCm][TunableOp] UTs, submatrix offline tuning, and Sca…
3f73e8a 
…ledGEMM rowwise fix (#2106)

Align TunableOp UTs, features, and bug fixes with upstream PyTorch main

UTs:
pytorch#148982
pytorch#149930
pytorch#150142
pytorch#150463

Feature: offline tuning for submatrices:
pytorch#151138

Bug Fix: ScaledGEMM rowwise
pytorch#152403

---------

Co-authored-by: Jeff Daily <jeff.daily@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffdaily jeffdaily jeffdaily approved these changes

@eqy eqy Awaiting requested review from eqy eqy is a code owner

@syed-ahmed syed-ahmed Awaiting requested review from syed-ahmed syed-ahmed is a code owner

@lezcano lezcano Awaiting requested review from lezcano lezcano is a code owner

@nikitaved nikitaved Awaiting requested review from nikitaved nikitaved is a code owner

@IvanYashchuk IvanYashchuk Awaiting requested review from IvanYashchuk IvanYashchuk is a code owner

Assignees

No one assigned

Labels

ciflow/rocm Trigger "default" config CI on ROCm ciflow/rocm-mi300 Trigger "default" config CI on ROCm MI300 ciflow/trunk Trigger trunk jobs on your pull request Merged module: rocm AMD GPU support for Pytorch open source topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@naromero77amd @pytorchmergebot @jeffdaily @pytorchbot