Install magma from a tarball by afrittoli · Pull Request #140417 · pytorch/pytorch

afrittoli · 2024-11-12T16:50:29Z

Magma is built for specific CUDA versions and stored in the ossci-linux bucket. Install it from there rather than the deprecated conda package.

There are two places where magma is installed today:

install_conda.sh: extract the magma package in the same exact location where conda would install it, using a dedicated install_magma_conda.sh script. The new script is included in the relevant Dockerfiles where CUDA+magma is needed
install_magma.sh: this script already uses a tarball. Use the new tarball instead of the tarball from the conda package. The format of the new tarball is compatible with the old one, so changes here are minimal:wq

Fixes #140538
Test PR: #141584

pytorch-bot · 2024-11-12T16:50:33Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/140417

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (4 Unrelated Failures)

As of commit cb46b69 with merge base 9012e7a ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / linux-focal-cuda12.4-py3.10-gcc9 / test (default, 1, 5, lf.linux.4xlarge.nvidia.gpu) (gh) (disabled by #141327 but the issue was closed recently and a rebase is needed to make it pass)
inductor/test_mkldnn_pattern_matcher.py::TestPatternMatcher::test_conv2d_binary_broadcast_shapes_cpu
pull / linux-focal-cuda12.4-py3.10-gcc9 / test (default, 4, 5, lf.linux.4xlarge.nvidia.gpu) (gh) (disabled by #141331 but the issue was closed recently and a rebase is needed to make it pass)
inductor/test_mkldnn_pattern_matcher.py::TestPatternMatcher::test_conv3d_binary_broadcast_shapes_cpu
pull / linux-focal-cuda12.4-py3.10-gcc9-sm86 / test (default, 2, 5, lf.linux.g5.4xlarge.nvidia.gpu) (gh) (disabled by #141331 but the issue was closed recently and a rebase is needed to make it pass)
inductor/test_mkldnn_pattern_matcher.py::TestPatternMatcher::test_conv3d_binary_broadcast_shapes_cpu
pull / linux-focal-py3.9-clang10 / test (default, 3, 5, lf.linux.4xlarge) (gh) (disabled by #141327 but the issue was closed recently and a rebase is needed to make it pass)
inductor/test_mkldnn_pattern_matcher.py::TestPatternMatcher::test_conv2d_binary_broadcast_shapes_cpu

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This is a test PR to validate the images produced by installing magma from tarball (instead of conda), introduced by pytorch#140417 Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>

afrittoli · 2024-11-26T21:17:00Z

@atalman Tested at #141584.
The s390x and onnx failures definitely seem unrelated.
I'm unsure about the linux-jammy-py3-clang12-executorch failure. WDYT?

atalman

lgtm

atalman · 2024-11-27T21:54:36Z

@pytorchmergebot merge -f "Failures are not related"

pytorchmergebot · 2024-11-27T21:56:04Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

atalman · 2024-11-27T23:21:17Z

@pytorchmergebot revert -c nosignal -m "Caused failures in calculate docker image"

pytorchmergebot · 2024-11-27T23:22:40Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

This reverts commit 30ab102. Reverted #140417 on behalf of https://github.com/atalman due to Caused failures in calculate docker image ([comment](#140417 (comment)))

pytorchmergebot · 2024-11-27T23:22:46Z

@afrittoli your PR has been successfully reverted.

This PR was reopened (likely due to being reverted), so your approval was removed. Please request another review.

atalman · 2024-11-27T23:42:21Z

@afrittoli sorry for revert. Will try to reland this tomorrow. Looks like the issue after landing this PR was caused by CI flakiness, not PR itself

atalman · 2024-11-28T14:32:49Z

@pytorchmergebot rebase -b main

pytorchmergebot · 2024-11-28T14:34:24Z

@pytorchbot started a rebase job onto refs/remotes/origin/main. Check the current status here

pytorchmergebot · 2024-11-28T14:34:26Z

Rebase failed due to

Aborting rebase because rebasing the branch resulted in the same sha as the target branch.
This usually happens because the PR has already been merged.  Please rebase locally and push.

Raised by https://github.com/pytorch/pytorch/actions/runs/12070880702

Magma is built for specific CUDA versions and stored in the ossci-linux bucket. Install it from there rather than the deprecated conda package. There are two places where magma is installed today: - install_conda.sh: extract the magma package in the same exact location where conda would install it, using a dedicated install_magma_conda.sh script - install_magma.sh: this script already uses a tarball. Use the new tarball instead of the tarball from the conda package. The format of the new tarball is compatible with the old one, so changes here are minimal:wq Signed-off-by: Andrea Frittoli <andrea.frittoli@uk.ibm.com>

atalman

lgtm

Magma is built for specific CUDA versions and stored in the ossci-linux bucket. Install it from there rather than the deprecated conda package. There are two places where magma is installed today: - `install_conda.sh`: extract the magma package in the same exact location where conda would install it, using a dedicated `install_magma_conda.sh` script. The new script is included in the relevant Dockerfiles where CUDA+magma is needed - `install_magma.sh`: this script already uses a tarball. Use the new tarball instead of the tarball from the conda package. The format of the new tarball is compatible with the old one, so changes here are minimal:wq Fixes pytorch#140538 Test PR: pytorch#141584 Pull Request resolved: pytorch#140417 Approved by: https://github.com/atalman

This reverts commit 30ab102. Reverted pytorch#140417 on behalf of https://github.com/atalman due to Caused failures in calculate docker image ([comment](pytorch#140417 (comment)))

atalman · 2024-12-05T15:18:56Z

@pytorchmergebot merge -f "All required jobs are green"

pytorchmergebot · 2024-12-05T15:20:47Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Magma is built for specific CUDA versions and stored in the ossci-linux bucket. Install it from there rather than the deprecated conda package. There are two places where magma is installed today: - `install_conda.sh`: extract the magma package in the same exact location where conda would install it, using a dedicated `install_magma_conda.sh` script. The new script is included in the relevant Dockerfiles where CUDA+magma is needed - `install_magma.sh`: this script already uses a tarball. Use the new tarball instead of the tarball from the conda package. The format of the new tarball is compatible with the old one, so changes here are minimal:wq Fixes pytorch#140538 Test PR: pytorch#141584 Pull Request resolved: pytorch#140417 Approved by: https://github.com/atalman

…ll (#147476) Related to: #138506 we migrated magma-cuda build from anaconda to aws Last version of magma-cuda published was 12.6 https://anaconda.org/pytorch/magma-cuda126 Here is the PR that moved from anaconda to tarball: #140417 Pull Request resolved: #147476 Approved by: https://github.com/albanD

afrittoli requested a review from jeffdaily as a code owner November 12, 2024 16:50

pytorch-bot Bot added the topic: not user facing topic category label Nov 12, 2024

afrittoli force-pushed the magma_from_tarball branch from 68d0d77 to 1bb97ee Compare November 12, 2024 16:50

pytorchbot added the open source label Nov 12, 2024

afrittoli force-pushed the magma_from_tarball branch 4 times, most recently from aad2e03 to a132112 Compare November 12, 2024 22:30

soulitzer requested a review from a team November 13, 2024 04:13

soulitzer added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Nov 13, 2024

afrittoli force-pushed the magma_from_tarball branch 2 times, most recently from 6f707b7 to e2b9fa0 Compare November 13, 2024 10:47

afrittoli mentioned this pull request Nov 13, 2024

Update build jobs that use magma-cuda to use the new package #140538

Closed

afrittoli mentioned this pull request Nov 26, 2024

DNM - PR to test new magma images #141584

Closed

atalman previously approved these changes Nov 26, 2024

View reviewed changes

pytorchmergebot added the merging label Nov 27, 2024

pytorchmergebot added the Merged label Nov 27, 2024

pytorchmergebot closed this in 30ab102 Nov 27, 2024

pytorchmergebot removed the merging label Nov 27, 2024

pytorchmergebot added the Reverted label Nov 27, 2024

pytorchmergebot added the ci-no-td Do not run TD on this PR label Nov 27, 2024

pytorchmergebot reopened this Nov 27, 2024

afrittoli mentioned this pull request Nov 28, 2024

Replace magma-cuda dependency on pytorch/conda-builder Docker image #138909

Closed

afrittoli force-pushed the magma_from_tarball branch from e2b9fa0 to f068b7d Compare November 28, 2024 15:06

afrittoli closed this Nov 28, 2024

afrittoli reopened this Nov 28, 2024

atalman mentioned this pull request Dec 2, 2024

Jobs failing on Calculate Docker image step when modifying Docker files #141885

Closed

afrittoli force-pushed the magma_from_tarball branch from f068b7d to cb46b69 Compare December 2, 2024 17:47

atalman approved these changes Dec 3, 2024

View reviewed changes

afrittoli mentioned this pull request Dec 4, 2024

DNM - Test docker builds with higher timeout #142060

Closed

pytorchmergebot added the merging label Dec 5, 2024

pytorchmergebot closed this in 82c1403 Dec 5, 2024

pytorchmergebot removed the merging label Dec 5, 2024

atalman mentioned this pull request Feb 19, 2025

Remove mention of magma-cuda in readme.md, refactor magma_conda install #147476

Closed

Conversation

afrittoli commented Nov 12, 2024 • edited by atalman Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/140417

✅ You can merge normally! (4 Unrelated Failures)

Uh oh!

afrittoli commented Nov 26, 2024

Uh oh!

atalman left a comment

Choose a reason for hiding this comment

Uh oh!

atalman commented Nov 27, 2024

Uh oh!

pytorchmergebot commented Nov 27, 2024

Merge started

Uh oh!

atalman commented Nov 27, 2024

Uh oh!

pytorchmergebot commented Nov 27, 2024

Uh oh!

pytorchmergebot commented Nov 27, 2024

Uh oh!

atalman commented Nov 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

atalman commented Nov 28, 2024

Uh oh!

pytorchmergebot commented Nov 28, 2024

Uh oh!

pytorchmergebot commented Nov 28, 2024

Uh oh!

atalman left a comment

Choose a reason for hiding this comment

Uh oh!

atalman commented Dec 5, 2024

Uh oh!

pytorchmergebot commented Dec 5, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

afrittoli commented Nov 12, 2024 •

edited by atalman

Loading

pytorch-bot Bot commented Nov 12, 2024 •

edited

Loading

atalman commented Nov 27, 2024 •

edited

Loading