The dist sender sends a batch to many ranges at once by chunking the batch up into one per range, and making a potentially asynchronous RPC (or local call, if the range lives on the local node) to each range.

Consider a case where there are 10 ranges per node, and a batch contains requests for all 10 ranges for a remote node. Then, the dist sender will make 10 asynchronous network RPCs all to the same node.

This seems inefficient. If the dist sender instead chunked up a batch by node, instead of by range, and sent a single RPC per node, then we'd have way fewer in-flight RPCs. The receiving node could then be in charge of further concurrency if it chooses. At the moment, the dist sender is the arbiter of concurrency - even for remote nodes, whose load it knows nothing about!

I'm assigning @nvanbenschoten for initial thoughts.