@xush6528 pointed out in #23968 that we should make send task non-blocking in ProcessGroupAgent. It currently waits until both preamble and payload send finishes. We could use a separate send GC thread that captures the preamble, payload and ProcessGroup::Work (all as std::shared_ptr) in a GC work, keep them in a queue, wait for completion in order, and destruct preamble and payload tensors.
cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera
@xush6528 pointed out in #23968 that we should make send task non-blocking in
ProcessGroupAgent. It currently waits until both preamble and payload send finishes. We could use a separate send GC thread that captures thepreamble,payloadandProcessGroup::Work(all asstd::shared_ptr) in a GC work, keep them in a queue, wait for completion in order, and destructpreambleandpayloadtensors.cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera