Skip to content

Conversation

@csegarragonz
Copy link
Collaborator

@csegarragonz csegarragonz commented Dec 1, 2021

In this PR I apply the same logic from #187 to the reduce collecive communication algorithm.

The logic is the same than for broadcast() but inverted.

  1. If we are the receiver of the reduce, we expect all our local ranks to send their data for reduction. We also expect all remote leaders to send us their data.
  2. If we are a local leader such that we are not co-located with the receiver rank: we (i) wait for all our local ranks to send us their data for reduction, and (ii) send our reduced data to the receiver of the reduce (1).
  3. If we are neither the receiver rank nor a local leader, we send our data for reduction to either our local master or the receiver: if we are colocated with the receiver, we send to the receiver, and if not we send to our local master.

Note that this assumes all our reduce operations to be associative and commutative, which is an assumption we already make.

@csegarragonz csegarragonz added the mpi Related to the MPI implementation label Dec 1, 2021
@csegarragonz csegarragonz self-assigned this Dec 1, 2021
@csegarragonz csegarragonz marked this pull request as draft December 1, 2021 17:39
@csegarragonz csegarragonz marked this pull request as ready for review December 1, 2021 17:57
@csegarragonz csegarragonz force-pushed the reduce-locality branch 3 times, most recently from 2de2eef to d19a76c Compare December 6, 2021 10:33
@csegarragonz csegarragonz merged commit 08381af into master Dec 6, 2021
@csegarragonz csegarragonz deleted the reduce-locality branch December 6, 2021 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mpi Related to the MPI implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants