[c10d] Make C10d support CPU only build#11513
Closed
teng-li wants to merge 3 commits intopytorch:masterfrom
Closed
[c10d] Make C10d support CPU only build#11513teng-li wants to merge 3 commits intopytorch:masterfrom
teng-li wants to merge 3 commits intopytorch:masterfrom
Conversation
apaszke
approved these changes
Sep 12, 2018
Contributor
apaszke
left a comment
There was a problem hiding this comment.
I'm not an expert in our build system nor c10d, but looks ok after a cursory look. I have some comments that could be improved. It's not very nice that we have to sprinkle that many ifdefs, and I'm not sure if that will work fine with our binaries, so it might be better to consult someone else too.
| endif() | ||
| else() | ||
| message(STATUS "Building C10D without CUDA support") | ||
| endif() |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| add_definitions(-DUSE_CUDA=1) | ||
| endif() | ||
| else() | ||
| message(STATUS "Building C10D without CUDA support") |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| #ifdef USE_CUDA | ||
| , | ||
| cuda_(false) | ||
| #endif |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| for (size_t i = 0; i < srcSizes.size(); i++) { | ||
| #ifdef USE_CUDA | ||
| deviceGuard.set_index(key.type->is_cuda() ? key.devices[i] : -1); | ||
| #else |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| c10d_add_test(ProcessGroupGlooAsyncTest.cpp c10d c10d_cuda_test) | ||
| if(DISTRIBUTED_NCCL_FOUND) | ||
| c10d_add_test(ProcessGroupNCCLTest.cpp c10d c10d_cuda_test) | ||
| endif() |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| if(USE_CUDA AND CUDA_FOUND) | ||
| cuda_add_library(c10d_cuda_test CUDATest.cu) | ||
| target_link_libraries(c10d_cuda_test c10d) | ||
| endif() |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
Contributor
facebook-github-bot
left a comment
There was a problem hiding this comment.
teng-li has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
petrex
pushed a commit
to petrex/pytorch
that referenced
this pull request
Sep 12, 2018
* master: (165 commits) Aibench for asr decoder Explicitly set locale on docs build. (pytorch#11595) Documentation for debugging JIT Fused weightnorm for ATen (pytorch#10842) Move Type, Tensor, TensorMethods to core. Add reminder % to the jit Fix reloading modules back into python (pytorch#11552) Add trigonometry functions to docs/source/onnx.rst Add EndToEndHybridModel CUDA tests (pytorch#11544) minor formatting error log (pytorch#11528) Warn that export+import module always load onto the CPU (pytorch#11485) caffe2::StorageImpl use at::DataPtr (pytorch#11282) Sync all libnccl soversions, not just libnccl.so.1 (pytorch#11575) Document BatchNorm and update default behavior (pytorch#11484) Typo fix in randomness.rst (pytorch#11571) Move some bmm/baddbmm to ATen (pytorch#11292) Make c10d test work on CPU only build (pytorch#11567) Clean up some C++ cruftiness in the script lexer. Allow setting deletion constant Make C10d support CPU only build (pytorch#11513) ...
laurentdupin
pushed a commit
to laurentdupin/pytorch
that referenced
this pull request
Apr 24, 2026
Summary: This makes torch.distributed works for CPU only build. Also added one more CI test case to cover MPI CPU build. All CI tests should cover this change Pull Request resolved: pytorch#11513 Differential Revision: D9784546 Pulled By: teng-li fbshipit-source-id: 0976a6b0fd199670926f0273e17ad7d2805e42e7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This makes torch.distributed works for CPU only build.
Also added one more CI test case to cover MPI CPU build.
All CI tests should cover this change