Skip to content

[CUDA][cuBLASLt] Fix scale setting for allowFP16AccumulationCuBLAS true case#153083

Closed
eqy wants to merge 3 commits intopytorch:mainfrom
eqy:fixscalefp16
Closed

[CUDA][cuBLASLt] Fix scale setting for allowFP16AccumulationCuBLAS true case#153083
eqy wants to merge 3 commits intopytorch:mainfrom
eqy:fixscalefp16

Conversation

@eqy
Copy link
Collaborator

@eqy eqy commented May 7, 2025 •
edited by pytorch-bot bot
Loading

Also add some missing @onlyCUDA / support check decorators in test_matmul_cuda.py
Should help resolve #151890

cc @ptrblck @msaroufim @jerryzh168 @csarofeen @xwang233

@eqy eqy requested a review from syed-ahmed as a code owner May 7, 2025 19:06
@eqy eqy added module: cuda Related to torch.cuda, and CUDA support in general module: cublas Problem related to cublas support open source module: half Related to float16 half-precision floats topic: not user facing topic category labels May 7, 2025
@pytorch-bot
Copy link

pytorch-bot bot commented May 7, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/153083

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 167c4a6 with merge base 590965f (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

janeyx99
janeyx99 reviewed May 7, 2025
View reviewed changes
janeyx99
janeyx99 reviewed May 7, 2025
View reviewed changes
aten/src/ATen/cuda/CUDABlas.cpp
cudaDeviceProp* prop = at::cuda::getCurrentDeviceProperties();
if (prop->major >= 7 && at::globalContext().allowFP16AccumulationCuBLAS()) {
computeType = CUBLAS_COMPUTE_16F;
scaleType = CUDA_R_16F;
Copy link
Contributor

@janeyx99 janeyx99 May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👀

janeyx99
janeyx99 approved these changes May 7, 2025
View reviewed changes
Copy link
Contributor

@janeyx99 janeyx99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok

@janeyx99 janeyx99 added release notes: cuda release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module and removed topic: not user facing topic category labels May 7, 2025
@eqy
Copy link
Collaborator Author

eqy commented May 8, 2025

@pytorchmergebot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label May 8, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented May 8, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented May 8, 2025

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch rebase origin/main returned non-zero exit code 1

Rebasing (1/1)
Auto-merging test/test_matmul_cuda.py
CONFLICT (content): Merge conflict in test/test_matmul_cuda.py
error: could not apply c5d1ed74a17... [CUDA][cuBLASLt] Fix scale setting for `allowFP16AccumulationCuBLAS` `true` case (#153083)
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Could not apply c5d1ed74a17... [CUDA][cuBLASLt] Fix scale setting for `allowFP16AccumulationCuBLAS` `true` case (#153083)
Details for Dev Infra team Raised by workflow job

@eqy
Copy link
Collaborator Author

eqy commented May 8, 2025

@pytorchmergebot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented May 8, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@janeyx99 janeyx99 janeyx99 approved these changes

@syed-ahmed syed-ahmed Awaiting requested review from syed-ahmed

Assignees

No one assigned

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged module: cublas Problem related to cublas support module: cuda Related to torch.cuda, and CUDA support in general module: half Related to float16 half-precision floats open source release notes: cuda release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

DISABLED test_cublas_and_lt_reduced_precision_fp16_accumulate_cuda (__main__.TestMatmulCudaCUDA)

3 participants

@eqy @pytorchmergebot @janeyx99