[Workflow][1/3] Remove benchmack tests from rerun disbled tests by yangw-dev · Pull Request #139337 · pytorch/pytorch

yangw-dev · 2024-10-30T23:24:20Z

Overview

Remove benchmark tests from rerun-disabled-tests, this is considered non-unittest.
See one page doc: [Bootcamp Task] Remove non-unittest test during rerun-disabled-tests

Manual Test

Test run Inductor.yml:
https://github.com/pytorch/pytorch/actions/runs/11603287758/job/32309968542?pr=139337
Test run inductor-unittest.yml (3cbd83d)
https://github.com/pytorch/pytorch/actions/runs/11605399925/job/32315737205?pr=139337

Steps to fix the issue

[THIS PR] Create inductor-unittest.yml to handle unit test and daily rerun for inductor
Create Inductor-cu124-unittest.yml to handle unit tests and daily rerun for inductor-cu124
Disable benchmark test in mixed test such as CPP_Wrapper which includes both unittest and benchmark test

pytorch-bot · 2024-10-30T23:24:24Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139337

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 4d1ec56 with merge base a426837 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / linux-focal-py3.9-clang10 / test (default, 3, 5, linux.4xlarge) (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

.github/workflows/inductor-unittest.yml

huydhn

LGTM!

yangw-dev · 2024-10-31T22:38:12Z

Test for input in workflow-level concurrency, echo event for:

  echo-event:
    runs-on: ubuntu-latest
    steps:
      - name: Echo GitHub Event
        env:
          GITHUB_EVENT: ${{ toJson(github.event) }}
        run: echo "$GITHUB_EVENT"

https://github.com/pytorch/pytorch/actions/runs/11620892828/job/32363610907?pr=139337
Seems like no inputs is in there

yangw-dev · 2024-11-01T05:58:42Z

@pytorchbot merge

pytorchmergebot · 2024-11-01T06:00:25Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…rch#139337) Fixes [pytorch#5774](pytorch/test-infra#5774) # Overview Remove benchmark tests from rerun-disabled-tests, this is considered non-unittest. See one page doc: [[Bootcamp Task] Remove non-unittest test during rerun-disabled-tests](https://docs.google.com/document/d/1xffkt_LNC5ZLsoVQDmuKbNqYnMUW_xYYStv66Pr-qac/edit?tab=t.0) # Manual Test - Test run Inductor.yml: https://github.com/pytorch/pytorch/actions/runs/11603287758/job/32309968542?pr=139337 - Test run inductor-unittest.yml ([3cbd83d](pytorch@3cbd83d)) https://github.com/pytorch/pytorch/actions/runs/11605399925/job/32315737205?pr=139337 # Steps to fix the issue - [x] [**THIS PR**] Create inductor-unittest.yml to handle unit test and daily rerun for inductor - [ ] Create Inductor-cu124-unittest.yml to handle unit tests and daily rerun for inductor-cu124 - [ ] Disable benchmark test in mixed test such as CPP_Wrapper which includes both unittest and benchmark test Pull Request resolved: pytorch#139337 Approved by: https://github.com/huydhn

This has been failing in trunk for sometimes, let's just update the accuracy results first. The command I run `python benchmarks/dynamo/ci_expected_accuracy/update_expected.py 127f836`. I also fix the update script a bit to make it working after #139337 Pull Request resolved: #144523 Approved by: https://github.com/kit1980, https://github.com/Skylion007

test

74c129a

pytorch-bot bot added the topic: not user facing topic category label Oct 30, 2024

yangw-dev added 9 commits October 30, 2024 16:27

test

66a0a72

test

6972bde

test

c3a1ec3

test

4555c87

test

3cbd83d

test

4befa38

test

37f0eef

test

c1b27c6

test

27df20e

yangw-dev changed the title ~~test~~ [Workflow] Remove benchmack tests Oct 31, 2024

yangw-dev changed the title ~~[Workflow] Remove benchmack tests~~ [Workflow] Remove benchmack tests from rerun disbled tests Oct 31, 2024

yangw-dev requested review from clee2000 and huydhn October 31, 2024 16:15

yangw-dev marked this pull request as ready for review October 31, 2024 16:16

yangw-dev requested a review from a team as a code owner October 31, 2024 16:16

yangw-dev added 3 commits October 31, 2024 09:17

typo

ea0f5cc

typo

73ca5fd

typo

ca20713

huydhn added the ciflow/inductor label Oct 31, 2024

typo

c44f34a

huydhn reviewed Oct 31, 2024

View reviewed changes

.github/workflows/inductor-unittest.yml Outdated Show resolved Hide resolved

typo

96b64f6

yangw-dev requested a review from huydhn October 31, 2024 17:36

yangw-dev marked this pull request as draft October 31, 2024 17:38

typo

94b8e0e

yangw-dev added ciflow/inductor and removed ciflow/inductor labels Oct 31, 2024

yangw-dev marked this pull request as ready for review October 31, 2024 18:20

huydhn reviewed Oct 31, 2024

View reviewed changes

.github/workflows/inductor-unittest.yml Outdated Show resolved Hide resolved

huydhn approved these changes Oct 31, 2024

View reviewed changes

yangw-dev added 3 commits October 31, 2024 12:06

test

00ebc89

test

aebbbe9

test

00acef6

yangw-dev removed the request for review from clee2000 October 31, 2024 19:33

yangw-dev added 8 commits October 31, 2024 12:35

test

1ed5b23

test

ee349a3

test

3fcff31

test

b827f3d

test

f8fa1fd

test

9f37772

test

1bc016a

test

80ef5a0

yangw-dev added 3 commits October 31, 2024 15:38

test

f57a513

test

4b4149e

test

4d1ec56

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Nov 1, 2024

pytorchmergebot added the merging label Nov 1, 2024

pytorchmergebot added the Merged label Nov 1, 2024

pytorchmergebot closed this in 307ee79 Nov 1, 2024

pytorchmergebot removed the merging label Nov 1, 2024

huydhn mentioned this pull request Jan 10, 2025

Update the accuracy results for moco and llama #144523

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Workflow][1/3] Remove benchmack tests from rerun disbled tests#139337

[Workflow][1/3] Remove benchmack tests from rerun disbled tests#139337
yangw-dev wants to merge 32 commits intopytorch:mainfrom
yangw-dev:addInductorRerun

yangw-dev commented Oct 30, 2024 •

edited

Loading

Uh oh!

pytorch-bot bot commented Oct 30, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

huydhn left a comment

Uh oh!

yangw-dev commented Oct 31, 2024

Uh oh!

yangw-dev commented Nov 1, 2024

Uh oh!

pytorchmergebot commented Nov 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

yangw-dev commented Oct 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Manual Test

Steps to fix the issue

Uh oh!

pytorch-bot bot commented Oct 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139337

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

Uh oh!

Uh oh!

huydhn left a comment

Choose a reason for hiding this comment

Uh oh!

yangw-dev commented Oct 31, 2024

Uh oh!

yangw-dev commented Nov 1, 2024

Uh oh!

pytorchmergebot commented Nov 1, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yangw-dev commented Oct 30, 2024 •

edited

Loading

pytorch-bot bot commented Oct 30, 2024 •

edited

Loading