Skip to content

[BE]: Update cusparselt to 0.7.1#155232

Closed
Skylion007 wants to merge 2 commits intopytorch:mainfrom
Skylion007:skylion007/update-cusparselt-0-7-1
Closed

[BE]: Update cusparselt to 0.7.1#155232
Skylion007 wants to merge 2 commits intopytorch:mainfrom
Skylion007:skylion007/update-cusparselt-0-7-1

Conversation

@Skylion007
Copy link
Collaborator

@Skylion007 Skylion007 commented Jun 5, 2025

Needed to support sparse operations on Blackwell, and implements new features for the library. Also optimizes library sizes vs 0.7

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Jun 5, 2025
@pytorch-bot
Copy link

pytorch-bot bot commented Jun 5, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155232

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 6e68752 with merge base da1f898 (image):

NEW FAILURES - The following jobs have failed:

UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@Skylion007 Skylion007 force-pushed the skylion007/update-cusparselt-0-7-1 branch from d78ec5a to 4686a22 Compare June 5, 2025 16:22
@Skylion007 Skylion007 requested review from albanD, atalman, eqy and nWEIdia June 5, 2025 16:41
@Skylion007 Skylion007 marked this pull request as ready for review June 5, 2025 16:47
@Skylion007 Skylion007 requested review from a team and jeffdaily as code owners June 5, 2025 16:47
@Skylion007 Skylion007 added the better-engineering Relatively self-contained tasks for better engineering contributors label Jun 5, 2025
@Skylion007 Skylion007 requested review from malfet and ngimel June 5, 2025 16:51
@Skylion007 Skylion007 force-pushed the skylion007/update-cusparselt-0-7-1 branch 2 times, most recently from cd8626d to e238d23 Compare June 6, 2025 16:13
@Skylion007
Copy link
Collaborator Author

Skylion007 commented Jun 6, 2025

@nWEIdia Any thoughts here?

@nWEIdia
Copy link
Collaborator

nWEIdia commented Jun 6, 2025

@nWEIdia Any thoughts here?

This may have implications (and potentially complications) on the binary size increase for the upcoming v2.8.
Could you please help figure out the size increase? It would also be great to be educated how the community is using the cusparselt library.
If the binary size increase is acceptable, I think we can merge this PR. But I think @tinglvv is also working on cuda 12.9.1 update, which may (or may not) affect the binary size. So I would prefer investigating this after cuda 12.9.1 update, especially on the binary size front.

cc @atalman @malfet @ptrblck @tinglvv

@Skylion007
Copy link
Collaborator Author

Skylion007 commented Jun 6, 2025 •
edited
Loading

@nWEIdia Any thoughts here?

This may have implications (and potentially complications) on the binary size increase for the upcoming v2.8. Could you please help figure out the size increase? It would also be great to be educated how the community is using the cusparselt library. If the binary size increase is acceptable, I think we can merge this PR. But I think @tinglvv is also working on cuda 12.9.1 update, which may (or may not) affect the binary size. So I would prefer investigating this after cuda 12.9.1 update, especially on the binary size front.

cc @atalman @malfet @ptrblck @tinglvv

We dynamically link cuSparseLT and NVidia distributes it as a separate wheel so it shouldn't have any signficant binary sizes increases in our wheel

@nWEIdia
Copy link
Collaborator

nWEIdia commented Jun 6, 2025

@nWEIdia Any thoughts here?

This may have implications (and potentially complications) on the binary size increase for the upcoming v2.8. Could you please help figure out the size increase? It would also be great to be educated how the community is using the cusparselt library. If the binary size increase is acceptable, I think we can merge this PR. But I think @tinglvv is also working on cuda 12.9.1 update, which may (or may not) affect the binary size. So I would prefer investigating this after cuda 12.9.1 update, especially on the binary size front.
cc @atalman @malfet @ptrblck @tinglvv

We dynamically link cuSparseLT and distribute it as a separate wheel so it shouldn't have any signficant binary sizes increases in our wheel

Ah yes, just recalled that separation part, thanks! In this case it LGTM.

@tinglvv
Copy link
Collaborator

tinglvv commented Jun 6, 2025

LGTM if CI is green. Will also upgrade cusparseLt for cuda 12.9 builds in my PRs.

@Skylion007
Copy link
Collaborator Author

Skylion007 commented Jun 6, 2025

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 6, 2025
@Skylion007 Skylion007 closed this Jun 6, 2025
@Skylion007 Skylion007 reopened this Jun 6, 2025
@Skylion007
Copy link
Collaborator Author

Skylion007 commented Jun 6, 2025

Just need @atalman to upload the binaries this one.

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jun 6, 2025

Merge failed

Reason: Approvers from one of the following sets are needed:

  • OSS CI (alband, dagitses, pytorch/pytorch-dev-infra)
  • superuser (pytorch/metamates)
  • Core Reviewers (mruberry, lezcano, Skylion007, ngimel, peterbell10, ...)
  • Core Maintainers (soumith, gchanan, ezyang, dzhulgakov, malfet, ...)
Details for Dev Infra team Raised by workflow job

Failing merge rule: Core Maintainers

@Skylion007
Copy link
Collaborator Author

Skylion007 commented Jun 7, 2025

@pytorchbot rebase

@Skylion007 Skylion007 added the ciflow/binaries_wheel Trigger binary build and upload jobs for wheel on the PR label Jun 7, 2025
@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jun 7, 2025

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jun 7, 2025

Successfully rebased skylion007/update-cusparselt-0-7-1 onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout skylion007/update-cusparselt-0-7-1 && git pull --rebase)

@pytorchmergebot pytorchmergebot force-pushed the skylion007/update-cusparselt-0-7-1 branch from e238d23 to eb24121 Compare June 7, 2025 14:28
Skylion007
Skylion007 commented Jun 8, 2025
View reviewed changes
.ci/manywheel/build_cuda.sh
'$ORIGIN/../../nvidia/curand/lib'
'$ORIGIN/../../nvidia/cusolver/lib'
'$ORIGIN/../../nvidia/cusparse/lib'
'$ORIGIN/../../nvidia/cusparselt/lib'
Copy link
Collaborator Author

@Skylion007 Skylion007 Jun 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nvidia moved the cusparselt library to under /nvidia/ from 0.6.3 to 0.7

@tinglvv tinglvv mentioned this pull request Jun 9, 2025
Closed
nWEIdia
nWEIdia reviewed Jun 9, 2025
View reviewed changes
@malfet
Copy link
Contributor

malfet commented Jun 9, 2025

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jun 9, 2025

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jun 9, 2025

Merge failed

Reason: 6 jobs have failed, first few of them are: linux-binary-manywheel / manywheel-py3_9-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_12-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_13-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_10-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_13t-rocm6_4-build / build

Details for Dev Infra team Raised by workflow job

@malfet
Copy link
Contributor

malfet commented Jun 9, 2025

@pytorchbot merge -i

@pytorchmergebot
Copy link
Collaborator

pytorchmergebot commented Jun 9, 2025

Merge started

Your change will be merged while ignoring the following 8 checks: docker-builds / docker-build (linux.12xlarge, pytorch-linux-jammy-py3-clang12-executorch), .github/workflows/pull.yml / linux-jammy-py3-clang12-executorch / build, linux-binary-manywheel / manywheel-py3_9-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_12-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_13-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_10-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_13t-rocm6_4-build / build, linux-binary-manywheel / manywheel-py3_11-rocm6_4-build / build

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nWEIdia nWEIdia nWEIdia approved these changes

@malfet malfet malfet approved these changes

@eqy eqy Awaiting requested review from eqy

@albanD albanD Awaiting requested review from albanD

@atalman atalman Awaiting requested review from atalman

@jeffdaily jeffdaily Awaiting requested review from jeffdaily jeffdaily is a code owner

@ngimel ngimel Awaiting requested review from ngimel

Assignees

No one assigned

Labels

better-engineering Relatively self-contained tasks for better engineering contributors ciflow/binaries_wheel Trigger binary build and upload jobs for wheel on the PR ciflow/trunk Trigger trunk jobs on your pull request Merged open source topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Skylion007 @nWEIdia @tinglvv @pytorchmergebot @malfet @pytorchbot