Skip to content

[Resubmission][Gradient Compression] Refactor default_hooks.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future#51400

Closed
wayi1 wants to merge 3 commits intogh/SciPioneer/51/basefrom
gh/SciPioneer/51/head
Closed

[Resubmission][Gradient Compression] Refactor default_hooks.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future#51400
wayi1 wants to merge 3 commits intogh/SciPioneer/51/basefrom
gh/SciPioneer/51/head

Conversation

@wayi1
Copy link
Copy Markdown
Contributor

@wayi1 wayi1 commented Jan 30, 2021 •
edited
Loading

Stack from ghstack:

Resubmission of #51094

Address #50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202

Differential Revision: D26162333

[Resubmission][Gradient Compression] Refactor default_hooks.py and po…
ea29d8e 
…werSGD_hook.py by creating a util function that make a vanilla allreduce future

Resubmission of #51094

Address #50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202

Differential Revision: [D26162333](https://our.internmc.facebook.com/intern/diff/D26162333/)

[ghstack-poisoned]
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Jan 30, 2021 •
edited
Loading

💊 CI failures summary and remediations

As of commit 95a8a14 (more details on the Dr. CI page):

  • 5/5 failures possibly* introduced in this PR
    • 3/5 non-CircleCI failure(s)

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_python_doc_build (1/1)

Step: "Doc Build and Push" (full log | diagnosis details | 🔁 rerun)

Jan 31 06:50:39 Makefile:38: recipe for target 'html' failed 
Jan 31 06:50:38 
Jan 31 06:50:38 copying static files... ... done
Jan 31 06:50:38 copying extra files... done
Jan 31 06:50:38 dumping search index in English (code: en)... done
Jan 31 06:50:38 dumping object inventory... done
Jan 31 06:50:38 build finished with problems, 1 warning.
Jan 31 06:50:38 /var/lib/jenkins/workspace/docs/src/pytorch-sphinx-theme/pytorch_sphinx_theme/search.html:21: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
Jan 31 06:50:38   <p class="last">
Jan 31 06:50:38 /var/lib/jenkins/workspace/docs/src/pytorch-sphinx-theme/pytorch_sphinx_theme/search.html:24: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
Jan 31 06:50:38   </p>
Jan 31 06:50:39 Makefile:38: recipe for target 'html' failed
Jan 31 06:50:39 make: *** [html] Error 1
Jan 31 06:50:39 ++ code=2
Jan 31 06:50:39 ++ '[' 2 -ne 0 ']'
Jan 31 06:50:39 ++ set +x
Jan 31 06:50:39 =========================
Jan 31 06:50:39 /var/lib/jenkins/workspace/docs/source/notes/broadcasting.rst:6: WARNING: 'any' reference target not found: numpy.doc.broadcasting
Jan 31 06:50:39 =========================
Jan 31 06:50:39 Docs build failed. If the failure is not clear, scan back in the log
Jan 31 06:50:39 for any WARNINGS or for the line build finished with problems
Jan 31 06:50:39 (tried to echo the WARNINGS above the ==== line)

1 failure not recognized by patterns:

Job Step Action
CircleCI pytorch_windows_vs2019_py36_cuda10.1_test1 Unknown 🔁 rerun

Extra GitHub checks: 1 failed

ci.pytorch.org: 1 failed

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

@facebook-github-bot facebook-github-bot added cla signed oncall: distributed Add this issue/PR to distributed oncall triage queue labels Jan 30, 2021
@wayi1 wayi1 mentioned this pull request Jan 30, 2021
Closed
Update on "[Resubmission][Gradient Compression] Refactor default_hook…
f08cd46 
…s.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future"

Resubmission of #51094

Address #50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202

Differential Revision: [D26162333](https://our.internmc.facebook.com/intern/diff/D26162333/)

[ghstack-poisoned]
wayi1 pushed a commit that referenced this pull request Jan 30, 2021
[Resubmission][Gradient Compression] Refactor default_hooks.py and po…
d3b0224 
…werSGD_hook.py by creating a util function that make a vanilla allreduce future

Pull Request resolved: #51400

Resubmission of #51094

Address #50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202
ghstack-source-id: 120715333

Differential Revision: [D26162333](https://our.internmc.facebook.com/intern/diff/D26162333/)
@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 30, 2021

Codecov Report

Merging #51400 (f08cd46) into gh/SciPioneer/51/base (dbfaf96) will decrease coverage by 0.33%.
The diff coverage is 20.00%.

@@                    Coverage Diff                    @@
##           gh/SciPioneer/51/base   #51400      +/-   ##
=========================================================
- Coverage                  80.86%   80.52%   -0.34%     
=========================================================
  Files                       1938     1938              
  Lines                     211184   211182       -2     
=========================================================
- Hits                      170767   170061     -706     
- Misses                     40417    41121     +704

Update on "[Resubmission][Gradient Compression] Refactor default_hook…
95a8a14 
…s.py and powerSGD_hook.py by creating a util function that make a vanilla allreduce future"

Resubmission of #51094

Address #50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression #47202

Differential Revision: [D26162333](https://our.internmc.facebook.com/intern/diff/D26162333/)

[ghstack-poisoned]
@wayi1 wayi1 mentioned this pull request Jan 31, 2021
Closed
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Feb 1, 2021

This pull request has been merged in 0831984.

@facebook-github-bot facebook-github-bot deleted the gh/SciPioneer/51/head branch February 5, 2021 15:21
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@facebook-github-bot
[Resubmission][Gradient Compression] Refactor default_hooks.py and po…
463ba23 
…werSGD_hook.py by creating a util function that make a vanilla allreduce future (pytorch#51400)

Summary:
Pull Request resolved: pytorch#51400

Resubmission of pytorch#51094

Address pytorch#50973 (comment)

Original PR issue: Investigate Applying PowerSGD to Communication Hook for Gradient Compression pytorch#47202
ghstack-source-id: 120725690

Test Plan:
buck test mode/dev-nosan caffe2/test/distributed:c10d -- test_powerSGD_ddp_comm_hook_nccl

buck test mode/dev-nosan caffe2/test/distributed:c10d -- test_default_ddp_comm_hooks_nccl

Reviewed By: rohan-varma

Differential Revision: D26162333

fbshipit-source-id: ccc2eae5383a23673e00d61cb5570fb8bf749cd0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mingzhe09088 mingzhe09088 Awaiting requested review from mingzhe09088

@mrshenli mrshenli Awaiting requested review from mrshenli

@pritamdamania87 pritamdamania87 Awaiting requested review from pritamdamania87

@zhaojuanmao zhaojuanmao Awaiting requested review from zhaojuanmao

1 more reviewer

@rohan-varma rohan-varma rohan-varma approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla signed Merged oncall: distributed Add this issue/PR to distributed oncall triage queue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wayi1 @facebook-github-bot @rohan-varma