Skip to content

[BE][CI] remove flakiness calculation in print_test_stats.py#79905

Closed
janeyx99 wants to merge 1 commit intopytorch:masterfrom
janeyx99:remove-flaky-calc
Closed

[BE][CI] remove flakiness calculation in print_test_stats.py#79905
janeyx99 wants to merge 1 commit intopytorch:masterfrom
janeyx99:remove-flaky-calc

Conversation

@janeyx99
Copy link
Copy Markdown
Contributor

@janeyx99 janeyx99 commented Jun 20, 2022

We now use Rockset to store test case stats and we can glean flakiness from that data, so this complicated logic isn't necessary anymore!

This will also prevent spurious red on trunk that occurs with this code e.g., https://github.com/pytorch/pytorch/runs/6968080598?check_suite_focus=true

test plan

print_test_stats work on the PR: https://github.com/pytorch/pytorch/runs/6974961237?check_suite_focus=true though it seems several jobs ran into HTTP 500 and Memory errors which I don't think is related to my code + more a problem with the server https://github.com/pytorch/pytorch/runs/6974938954?check_suite_focus=true

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Jun 20, 2022

🔗 Helpful links

✅ No Failures (1 Pending)

As of commit 29a5f5d (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚


This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

@janeyx99 janeyx99 force-pushed the remove-flaky-calc branch from f2bae7c to 29a5f5d Compare June 20, 2022 22:18
@janeyx99 janeyx99 requested a review from a team June 21, 2022 14:32
Copy link
Copy Markdown
Contributor

@zengk95 zengk95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown
Member

@seemethere seemethere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete it all!

@janeyx99
Copy link
Copy Markdown
Contributor Author

@pytorchbot merge -g

@pytorchmergebot
Copy link
Copy Markdown
Collaborator

@pytorchbot successfully started a merge job. Check the current status here

@pytorchmergebot
Copy link
Copy Markdown
Collaborator

@janeyx99 your PR has been successfully merged.

@github-actions
Copy link
Copy Markdown
Contributor

Hey @janeyx99.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

facebook-github-bot pushed a commit that referenced this pull request Jun 22, 2022
…#79905)

Summary:
We now use Rockset to store test case stats and we can glean flakiness from that data, so this complicated logic isn't necessary anymore!

This will also prevent spurious red on trunk that occurs with this code e.g., https://github.com/pytorch/pytorch/runs/6968080598?check_suite_focus=true

## test plan
print_test_stats work on the PR: https://github.com/pytorch/pytorch/runs/6974961237?check_suite_focus=true though it seems several jobs ran into HTTP 500 and Memory errors which I don't think is related to my code + more a problem with the server https://github.com/pytorch/pytorch/runs/6974938954?check_suite_focus=true

Pull Request resolved: #79905
Approved by: https://github.com/zengk95, https://github.com/seemethere

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/4f08d56183a19518dba65d692f9dd23d7308fbdb

Reviewed By: atalman

Differential Revision: D37327501

Pulled By: janeyx99

fbshipit-source-id: c55e4d027bd1a2ed3f840a51f31f78c94b014fac
miladm pushed a commit to miladm/pytorch that referenced this pull request Jun 27, 2022
…#79905)

We now use Rockset to store test case stats and we can glean flakiness from that data, so this complicated logic isn't necessary anymore!

This will also prevent spurious red on trunk that occurs with this code e.g., https://github.com/pytorch/pytorch/runs/6968080598?check_suite_focus=true

## test plan
print_test_stats work on the PR: https://github.com/pytorch/pytorch/runs/6974961237?check_suite_focus=true though it seems several jobs ran into HTTP 500 and Memory errors which I don't think is related to my code + more a problem with the server https://github.com/pytorch/pytorch/runs/6974938954?check_suite_focus=true

Pull Request resolved: pytorch#79905
Approved by: https://github.com/zengk95, https://github.com/seemethere
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
…#79905)

We now use Rockset to store test case stats and we can glean flakiness from that data, so this complicated logic isn't necessary anymore!

This will also prevent spurious red on trunk that occurs with this code e.g., https://github.com/pytorch/pytorch/runs/6968080598?check_suite_focus=true

## test plan
print_test_stats work on the PR: https://github.com/pytorch/pytorch/runs/6974961237?check_suite_focus=true though it seems several jobs ran into HTTP 500 and Memory errors which I don't think is related to my code + more a problem with the server https://github.com/pytorch/pytorch/runs/6974938954?check_suite_focus=true

Pull Request resolved: pytorch#79905
Approved by: https://github.com/zengk95, https://github.com/seemethere
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants