Skip to content

[ci] remove remaining RDS dependency#79370

Closed
suo wants to merge 8 commits intogh/suo/560/basefrom
gh/suo/560/head
Closed

[ci] remove remaining RDS dependency#79370
suo wants to merge 8 commits intogh/suo/560/basefrom
gh/suo/560/head

Conversation

@suo
Copy link
Copy Markdown
Member

@suo suo commented Jun 12, 2022

Stack from ghstack (oldest at bottom):

With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Jun 12, 2022

🔗 Helpful links

❌ 1 New Failures

As of commit 4b9dade (more details on the Dr. CI page):

Expand to see more
  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages

See GitHub Actions build pull / linux-focal-py3.7-gcc7 / test (backwards_compat, 1, 1, linux.2xlarge) (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

2022-06-15T17:18:23.8136066Z RuntimeError:
2022-06-15T17:18:23.1543903Z Author: PyTorch Team
2022-06-15T17:18:23.1544165Z Author-email: packages@pytorch.org
2022-06-15T17:18:23.1544398Z License: BSD-3
2022-06-15T17:18:23.1544705Z Location: /opt/conda/lib/python3.7/site-packages
2022-06-15T17:18:23.1544972Z Requires: typing-extensions
2022-06-15T17:18:23.1545185Z Required-by: 
2022-06-15T17:18:23.1903130Z + python check_forward_backward_compatibility.py --existing-schemas nightly_schemas.txt
2022-06-15T17:18:23.8135027Z Traceback (most recent call last):
2022-06-15T17:18:23.8135584Z   File "check_forward_backward_compatibility.py", line 345, in <module>
2022-06-15T17:18:23.8135880Z     s = parse_schema(line.strip())
2022-06-15T17:18:23.8136066Z RuntimeError: 
2022-06-15T17:18:23.8136352Z Unknown custom class type c10d.ProcessGroup. Please ensure it is registered.:
2022-06-15T17:18:23.8136982Z c10d::broadcast(__torch__.torch.classes.c10d.ProcessGroup _0, Tensor[] _1, int _2, int _3, int _4) -> __torch__.torch.classes.c10d.Work _0
2022-06-15T17:18:23.8137356Z                                              ~~~~~~~~~~~~ <--- HERE
2022-06-15T17:18:23.8137729Z 
2022-06-15T17:18:23.9303708Z + cleanup
2022-06-15T17:18:23.9304010Z + retcode=1
2022-06-15T17:18:23.9304265Z + set +x
2022-06-15T17:18:23.9340333Z ##[error]Process completed with exit code 1.
2022-06-15T17:18:23.9375074Z Prepare all required actions
2022-06-15T17:18:23.9375357Z Getting action download info

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
suo added 2 commits June 12, 2022 10:20
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
suo added a commit that referenced this pull request Jun 12, 2022
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

ghstack-source-id: a1f9fcb
Pull Request resolved: #79370
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
suo added a commit that referenced this pull request Jun 12, 2022
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

ghstack-source-id: 2654da6
Pull Request resolved: #79370
Comment thread test/test_import_stats.py
# as a way to track the duration of `import torch` in our ossci-metrics
# S3 bucket (see tools/stats/print_test_stats.py)
class TestImportTime(TestCase):
def test_time_import_torch(self):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be a good time now to set this to fail if it's over a threshold? Do we...care?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, if we do care about tracking these stats, we can just run these as normal, right? The test_runs in Rockset will keep track of these times already.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a good point! I will re-add

With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

[ghstack-poisoned]
@suo suo requested a review from janeyx99 June 13, 2022 16:21
facebook-github-bot pushed a commit that referenced this pull request Jun 15, 2022
Summary:
With #79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

Pull Request resolved: #79370

Approved by: https://github.com/janeyx99

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/964d5059587c017b02f1d7d62587722a38683c63

Reviewed By: malfet

Differential Revision: D37153120

Pulled By: suo

fbshipit-source-id: 6a2e1fb827a88e0828b1f05fdbe62f07ec61da87
@facebook-github-bot
Copy link
Copy Markdown
Contributor

@pytorchbot revert -m="Diff reverted internally" -c="ghfirst"

This Pull Request has been reverted by a revert inside Meta. To re-land this change, please open another pull request, assign the same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).)

@pytorchmergebot
Copy link
Copy Markdown
Collaborator

@pytorchbot successfully started a revert job. Check the current status here

pytorchmergebot added a commit that referenced this pull request Jun 15, 2022
This reverts commit 964d505.

Reverted #79370 on behalf of https://github.com/facebook-github-bot due to Diff reverted internally
@suo suo reopened this Jun 15, 2022
@suo suo closed this Jun 16, 2022
@facebook-github-bot facebook-github-bot deleted the gh/suo/560/head branch June 19, 2022 14:16
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
With pytorch#79366, the only remaining
use case for RDS is import latency stats. Afaik we do not look at these,
so removing, which eliminates our last dependency on RDS.

Pull Request resolved: pytorch#79370

Approved by: https://github.com/janeyx99
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
This reverts commit 5d70521.

Reverted pytorch#79370 on behalf of https://github.com/facebook-github-bot due to Diff reverted internally
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants