Skip to content

[Gradient Compression] Directly let world_size = group_to_use.size()#49715

Closed
wayi1 wants to merge 2 commits intogh/SciPioneer/42/basefrom
gh/SciPioneer/42/head
Closed

[Gradient Compression] Directly let world_size = group_to_use.size()#49715
wayi1 wants to merge 2 commits intogh/SciPioneer/42/basefrom
gh/SciPioneer/42/head

Conversation

@wayi1
Copy link
Copy Markdown
Contributor

@wayi1 wayi1 commented Dec 21, 2020 •
edited
Loading

Stack from ghstack:

Address the comment on #49417 (comment)

Differential Revision: D25673997

@wayi1 wayi1 mentioned this pull request Dec 21, 2020
Closed
@facebook-github-bot facebook-github-bot added the oncall: distributed Add this issue/PR to distributed oncall triage queue label Dec 21, 2020
@wayi1 wayi1 mentioned this pull request Dec 21, 2020
Closed
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Dec 21, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit 62e1ee7 (more details on the Dr. CI page):

  • 3/4 failures possibly* introduced in this PR
    • 1/3 non-CircleCI failure(s)
  • 1/4 broken upstream at merge base b288aee on Dec 21 from 3:35pm to 9:32pm

🕵️ 2 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_build (1/2)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result. 
  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git reset --hard 62e1ee76dc8b95350ecdc5018b203efff2d78aa5
HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git merge --allow-unrelated-histories --no-edit --no-ff 7b4a7661d6de659c8423015a2f3e93308eb83850
Auto-merging torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
CONFLICT (content): Merge conflict in torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1

See CircleCI build pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc5_4_build (2/2)

Step: "(Optional) Merge target branch" (full log | diagnosis details | 🔁 rerun)

Automatic merge failed; fix conflicts and then commit the result. 
  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git reset --hard 62e1ee76dc8b95350ecdc5018b203efff2d78aa5
HEAD is now at 62e1ee76dc Update on "[Gradient Compression] Directly let world_size = group_to_use.size()"
+ git merge --allow-unrelated-histories --no-edit --no-ff 7b4a7661d6de659c8423015a2f3e93308eb83850
Auto-merging torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
CONFLICT (content): Merge conflict in torch/distributed/algorithms/ddp_comm_hooks/powerSGD_hook.py
Automatic merge failed; fix conflicts and then commit the result.


Exited with code exit status 1

1 job timed out:

  • pytorch_linux_bionic_py3_8_gcc9_coverage_test1

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 8 times.

@wayi1 wayi1 mentioned this pull request Dec 21, 2020
Closed
wayi1 pushed a commit that referenced this pull request Dec 21, 2020
[Gradient Compression] Directly let world_size = process_group.size()
4d551bc 
Address the comment on #49417 (comment)

Differential Revision: [D25673997](https://our.internmc.facebook.com/intern/diff/D25673997/)

ghstack-source-id: 119021459
Pull Request resolved: #49715
@wayi1 wayi1 mentioned this pull request Dec 21, 2020
Closed
rohan-varma
rohan-varma reviewed Dec 22, 2020
View reviewed changes
Comment thread torch/distributed/algorithms/ddp_comm_hooks/default_hooks.py Outdated
@wayi1 wayi1 changed the title [Gradient Compression] Directly let world_size = process_group.size() [Gradient Compression] Directly let world_size = group_to_use.size() Dec 22, 2020
Update on "[Gradient Compression] Directly let world_size = group_to_…
62e1ee7 
…use.size()"


Address the comment on #49417 (comment)

Differential Revision: [D25673997](https://our.internmc.facebook.com/intern/diff/D25673997/)

[ghstack-poisoned]
wayi1 pushed a commit that referenced this pull request Dec 22, 2020
[Gradient Compression] Directly let world_size = group_to_use.size()
f33b442 
Pull Request resolved: #49715

Address the comment on #49417 (comment)
ghstack-source-id: 119049598

Differential Revision: [D25673997](https://our.internmc.facebook.com/intern/diff/D25673997/)
@wayi1 wayi1 requested a review from rohan-varma December 22, 2020 18:52
rohan-varma
rohan-varma approved these changes Dec 23, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@rohan-varma rohan-varma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, although might want to rebase since some CI looks like it failed with merge conflicts.

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Dec 23, 2020

This pull request has been merged in 55b431b.

@facebook-github-bot facebook-github-bot deleted the gh/SciPioneer/42/head branch December 26, 2020 15:18
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@facebook-github-bot
[Gradient Compression] Directly let world_size = group_to_use.size() (p…
9fea8e8 
…ytorch#49715)

Summary:
Pull Request resolved: pytorch#49715

Address the comment on pytorch#49417 (comment)
ghstack-source-id: 119049598

Test Plan: waitforbuildbot

Reviewed By: rohan-varma

Differential Revision: D25673997

fbshipit-source-id: 44eb2540e5a77331c34ba503285cbd0bd63c2c0a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mingzhe09088 mingzhe09088 Awaiting requested review from mingzhe09088

@mrshenli mrshenli Awaiting requested review from mrshenli

@pritamdamania87 pritamdamania87 Awaiting requested review from pritamdamania87

@zhaojuanmao zhaojuanmao Awaiting requested review from zhaojuanmao

1 more reviewer

@rohan-varma rohan-varma rohan-varma approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla signed Merged oncall: distributed Add this issue/PR to distributed oncall triage queue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wayi1 @facebook-github-bot @rohan-varma