Skip to content

Avoid nested CommTensor wrapping#84963

Closed
mrshenli wants to merge 1 commit intogh/mrshenli/334/basefrom
gh/mrshenli/334/head
Closed

Avoid nested CommTensor wrapping#84963
mrshenli wants to merge 1 commit intogh/mrshenli/334/basefrom
gh/mrshenli/334/head

Conversation

@mrshenli
Copy link
Copy Markdown
Contributor

@mrshenli mrshenli commented Sep 13, 2022 •
edited
Loading

Stack from ghstack (oldest at bottom):

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot bot commented Sep 13, 2022 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/84963

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures, 11 Pending

As of commit 61e89db:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Sep 13, 2022
wanchaol
wanchaol approved these changes Sep 13, 2022
View reviewed changes
Copy link
Copy Markdown
Collaborator

@wanchaol wanchaol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have one comment about testing

test/distributed/test_c10d_common.py
work = dist.all_reduce(CommTensor(tensor), group=group, async_op=True)
return work, tensor

self._test_work_wait(tensor, comm_fn=comm_fn)
Copy link
Copy Markdown
Collaborator

@wanchaol wanchaol Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we create a CommTensor here before passing to comm_fn to verify the changes?

@facebook-github-bot facebook-github-bot added the oncall: distributed Add this issue/PR to distributed oncall triage queue label Sep 14, 2022
@mrshenli
Copy link
Copy Markdown
Contributor Author

mrshenli commented Sep 14, 2022

@pytorchbot merge -g

@pytorchmergebot
Copy link
Copy Markdown
Collaborator

pytorchmergebot commented Sep 14, 2022

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the green (-g) flag. This means that your change will be merged once all checks on your PR have passed (ETA: 0-4 Hours). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

@mrshenli mrshenli mentioned this pull request Sep 14, 2022
Merged
This was referenced Sep 14, 2022
Closed
Closed
mehtanirav pushed a commit that referenced this pull request Oct 4, 2022
@mrshenli @mehtanirav
Avoid nested CommTensor wrapping (#84963)
f2b3aee 
Pull Request resolved: #84963
Approved by: https://github.com/wanchaol
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wanchaol wanchaol wanchaol approved these changes

@pritamdamania87 pritamdamania87 Awaiting requested review from pritamdamania87

@zhaojuanmao zhaojuanmao Awaiting requested review from zhaojuanmao

@rohan-varma rohan-varma Awaiting requested review from rohan-varma

@H-Huang H-Huang Awaiting requested review from H-Huang

@awgu awgu Awaiting requested review from awgu

@mingzhe09088 mingzhe09088 Awaiting requested review from mingzhe09088

Assignees

No one assigned

Labels

cla signed Merged oncall: distributed Add this issue/PR to distributed oncall triage queue topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mrshenli @pytorchmergebot @wanchaol @facebook-github-bot