Setup TorchBench in Docker by huydhn · Pull Request #158613 · pytorch/pytorch

huydhn · 2025-07-18T00:21:08Z

This reduces the time spending to setup TorchBench in A100/H100 by another half an hour

Testing

H100 benchmark https://github.com/pytorch/pytorch/actions/runs/16396172453. Once this done, I will review the results on HUD to confirm that all models are there
A100 benchmark https://github.com/pytorch/pytorch/actions/runs/16396173932

cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @Lucaskabela

[ghstack-poisoned]

huydhn · 2025-07-18T00:21:09Z

Stack from ghstack (oldest at bottom):

-> Setup TorchBench in Docker #158613

pytorch-bot · 2025-07-18T00:21:12Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158613

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 94fa1bf with merge base 6100ed4 ():

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

pull / linux-jammy-py3_9-clang9-xla / test (xla, 1, 1, linux.12xlarge, unstable) (gh) (#158876)
sccache: error: couldn't connect to server

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This reduces the time spending to setup TorchBench in A100/H100 by half an hour Signed-off-by: Huy Do <huydhn@gmail.com> ghstack-source-id: 7425aa2 ghstack-comment-id: 3086043340 Pull-Request: #158613

[ghstack-poisoned]

This reduces the time spending to setup TorchBench in A100/H100 by half an hour Signed-off-by: Huy Do <huydhn@gmail.com> ghstack-source-id: 9eaa733 ghstack-comment-id: 3086043340 Pull-Request: #158613 Signed-off-by: Huy Do <huydhn@gmail.com>

[ghstack-poisoned]

This reduces the time spending to setup TorchBench in A100/H100 by half an hour Signed-off-by: Huy Do <huydhn@gmail.com> ghstack-source-id: 536ab13 ghstack-comment-id: 3086043340 Pull-Request: #158613 Signed-off-by: Huy Do <huydhn@gmail.com>

[ghstack-poisoned]

This reduces the time spending to setup TorchBench in A100/H100 by another half an hour Signed-off-by: Huy Do <huydhn@gmail.com> ghstack-source-id: c3c6d5b ghstack-comment-id: 3086043340 Pull-Request: #158613 Signed-off-by: Huy Do <huydhn@gmail.com>

[ghstack-poisoned]

This reverts commit cab2833. Reverted #158613 on behalf of https://github.com/ZainRizvi due to Seems to have broken trunk. See [GH job link](https://github.com/pytorch/pytorch/actions/runs/16429779764/job/46430634676) [HUD commit link](https://hud.pytorch.org/pytorch/pytorch/commit/b3c868d603e8f7b6661c93cd3d50c9a7b213ad6c) ([comment](#158613 (comment)))

This reduces the time spending to setup TorchBench in A100/H100 by another half an hour ### Testing * H100 benchmark https://github.com/pytorch/pytorch/actions/runs/16396172453. Once this done, I will review the results on [HUD](https://hud.pytorch.org/benchmark/compilers?dashboard=torchinductor&startTime=Fri%2C%2011%20Jul%202025%2023%3A01%3A24%20GMT&stopTime=Fri%2C%2018%20Jul%202025%2023%3A01%3A24%20GMT&granularity=hour&mode=inference&dtype=bfloat16&deviceName=cuda%20(h100)&lBranch=gh/huydhn/6/head&lCommit=14a38c719b29a19f518239b5edb084838ac5d2fb&rBranch=main&rCommit=0a99b026d6bd0f67dc2c0a20fe3228ddc4144854) to confirm that all models are there * A100 benchmark https://github.com/pytorch/pytorch/actions/runs/16396173932 Signed-off-by: Huy Do <huydhn@gmail.com> Pull Request resolved: pytorch#158613 Approved by: https://github.com/janeyx99

This reverts commit cab2833. Reverted pytorch#158613 on behalf of https://github.com/ZainRizvi due to Seems to have broken trunk. See [GH job link](https://github.com/pytorch/pytorch/actions/runs/16429779764/job/46430634676) [HUD commit link](https://hud.pytorch.org/pytorch/pytorch/commit/b3c868d603e8f7b6661c93cd3d50c9a7b213ad6c) ([comment](pytorch#158613 (comment)))

Signed-off-by: Huy Do <huydhn@gmail.com>

huydhn · 2025-07-26T19:37:34Z

@pytorchbot merge

pytorchmergebot · 2025-07-26T19:39:14Z

Merge failed

Reason: PR 158613 is out of sync with the corresponding revision fa565ea on branch gh/huydhn/6/orig that would be merged into main. This usually happens because there is a non ghstack change in the PR. Please sync them and try again (ex. make the changes on origin/gh/huydhn/6/orig and run ghstack).

Details for Dev Infra team

Raised by workflow job

pytorchmergebot · 2025-07-26T19:49:45Z

Merge failed

Reason: PR 158613 is out of sync with the corresponding revision fa565ea on branch gh/huydhn/6/orig that would be merged into main. This usually happens because there is a non ghstack change in the PR. Please sync them and try again (ex. make the changes on origin/gh/huydhn/6/orig and run ghstack).

Details for Dev Infra team

Raised by workflow job

huydhn · 2025-07-26T19:51:46Z

@pytorchbot merge

pytorchmergebot · 2025-07-26T19:53:20Z

Merge failed

Reason: Found an unexpected number of PRs mentioned in commit 0e37fc8: 0. This is probably because you are using an old version of ghstack. Please update ghstack and resubmit your PRs

Details for Dev Infra team

Raised by workflow job

huydhn · 2025-07-26T19:55:48Z

I screwed up my ghstack, but as the PR has been approved and all the signal are ok, I'm going to merge the change

XuehaiPan · 2025-07-28T06:17:16Z

@pytorchbot revert -c nosignal -m "checkout_install_torchbench function is removed but still referenced in trunk"

https://github.com/pytorch/pytorch/actions/runs/16548440338

pytorch/.ci/pytorch/macos-test.sh

Line 184 in 1cffb21

checkout_install_torchbench

pytorchmergebot · 2025-07-28T06:18:48Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2025-07-28T06:19:03Z

@huydhn your PR has been successfully reverted.

This reverts commit d72ebef. Reverted #158613 on behalf of https://github.com/XuehaiPan due to checkout_install_torchbench function is removed but still referenced in trunk ([comment](#158613 (comment)))

huydhn added 8 commits July 17, 2025 15:51

Update

0e37fc8

[ghstack-poisoned]

Update

ec8fa3c

[ghstack-poisoned]

Update

3aec24e

[ghstack-poisoned]

Update

30c21cb

[ghstack-poisoned]

Update

444df0f

[ghstack-poisoned]

Update

55316ba

[ghstack-poisoned]

Update

a151d80

[ghstack-poisoned]

Update

0ce8369

[ghstack-poisoned]

huydhn requested review from a team and jeffdaily as code owners July 18, 2025 00:21

pytorch-bot Bot added ciflow/inductor release notes: releng release notes category labels Jul 18, 2025

This was referenced Jul 18, 2025

Use linux.12xlarge.memory to build for H100/sm_90 #158598

Closed

Clean up some unused build env variables #158599

Closed

Build domain libraries on the build job #158600

Closed

Build domain libraries for all workflows with TorchBench config #158601

Closed

Update

9fe4955

[ghstack-poisoned]

huydhn added the no-runner-experiments Bypass Meta/LF runner determinator label Jul 18, 2025

huydhn added the test-config/default label Jul 18, 2025

huydhn added 3 commits July 17, 2025 18:14

Update

9ff8af5

[ghstack-poisoned]

Update

753fe8c

[ghstack-poisoned]

Update

2435a78

[ghstack-poisoned]

Update

2bc4a7d

[ghstack-poisoned]

Update

14a38c7

[ghstack-poisoned]

pytorchmergebot added the ci-no-td Do not run TD on this PR label Jul 22, 2025

pytorchmergebot reopened this Jul 22, 2025

huydhn removed the test-config/default label Jul 22, 2025

huydhn added 3 commits July 22, 2025 16:48

Merge branch 'main' into gh/huydhn/6/head

37ff176

Fix CI permission issues

a0aa37a

Signed-off-by: Huy Do <huydhn@gmail.com>

One last failure

94fa1bf

Signed-off-by: Huy Do <huydhn@gmail.com>

huydhn removed ci-no-td Do not run TD on this PR oncall: distributed Add this issue/PR to distributed oncall triage queue labels Jul 26, 2025

pytorchmergebot added the merging label Jul 26, 2025

pytorchmergebot added merging and removed merging labels Jul 26, 2025

pytorchmergebot removed the merging label Jul 26, 2025

pytorchmergebot added the merging label Jul 26, 2025

pytorchmergebot removed the merging label Jul 26, 2025

huydhn merged commit d72ebef into main Jul 26, 2025
256 of 261 checks passed

pytorchmergebot added the ci-no-td Do not run TD on this PR label Jul 28, 2025

huydhn mentioned this pull request Jul 28, 2025

[Take 2] Setup TorchBench in Docker #159300

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setup TorchBench in Docker#158613

Setup TorchBench in Docker#158613
huydhn merged 29 commits intomainfrom
gh/huydhn/6/head

huydhn commented Jul 18, 2025 •

edited

Loading

Uh oh!

huydhn commented Jul 18, 2025 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Jul 18, 2025 •

edited

Loading

Uh oh!

huydhn commented Jul 26, 2025

Uh oh!

pytorchmergebot commented Jul 26, 2025

Uh oh!

pytorchmergebot commented Jul 26, 2025

Uh oh!

huydhn commented Jul 26, 2025

Uh oh!

pytorchmergebot commented Jul 26, 2025

Uh oh!

huydhn commented Jul 26, 2025

Uh oh!

Uh oh!

XuehaiPan commented Jul 28, 2025

Uh oh!

pytorchmergebot commented Jul 28, 2025

Uh oh!

pytorchmergebot commented Jul 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

huydhn commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Uh oh!

huydhn commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158613

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

huydhn commented Jul 26, 2025

Uh oh!

pytorchmergebot commented Jul 26, 2025

Merge failed

Uh oh!

pytorchmergebot commented Jul 26, 2025

Merge failed

Uh oh!

huydhn commented Jul 26, 2025

Uh oh!

pytorchmergebot commented Jul 26, 2025

Merge failed

Uh oh!

huydhn commented Jul 26, 2025

Uh oh!

Uh oh!

XuehaiPan commented Jul 28, 2025

Uh oh!

pytorchmergebot commented Jul 28, 2025

Uh oh!

pytorchmergebot commented Jul 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

huydhn commented Jul 18, 2025 •

edited

Loading

huydhn commented Jul 18, 2025 •

edited

Loading

pytorch-bot Bot commented Jul 18, 2025 •

edited

Loading