Type of Change
Bug fix
Problem Statement
##Summary
When using disaggregated prefill/decode with DP attention mode, the current implementation couples the decode routing decision with the KV transfer source identification. This prevents independent DP size configurations between prefill and decode servers and limits routing flexibility.
I think we can support this in Dynamo itself by being clever about the boostrap room
An idea: align bootstrap_room with Prefill DP Rank (doesnt require sglang changes)
Generate bootstrap_room values that encode the chosen prefill DP rank:
// In prefill_router.rs
// Current (sequential):
// let bootstrap_room = BOOTSTRAP_ROOM_COUNTER.fetch_add(1, Ordering::Relaxed);
// Proposed (encodes prefill dp_rank):
let prefill_dp_size = get_prefill_dp_size(); // From worker registration
let base = BOOTSTRAP_ROOM_COUNTER.fetch_add(prefill_dp_size, Ordering::Relaxed);
let bootstrap_room = base + chosen_prefill_dp_rank;
Flow would look like
Dynamo KV router picks prefill DP rank 2
│
▼
bootstrap_room = base + 2 (guarantees bootstrap_room % dp_size == 2)
│
├──► Prefill routing: Dynamo routes to DP rank 2
│
▼
Decode request (no dp_rank set)
│
├──► Decode routing: Dynamo KV router picks independently (e.g., DP rank 0)
│
└──► KV receiver: prefill_dp_rank = bootstrap_room % prefill_dp_size = 2 ✓
Proposed Solution
.
Estimated PR Size
XS (1-10 lines)
Files/Components Affected
Would close DYN-1841
Type of Change
Bug fix
Problem Statement
##Summary
When using disaggregated prefill/decode with DP attention mode, the current implementation couples the decode routing decision with the KV transfer source identification. This prevents independent DP size configurations between prefill and decode servers and limits routing flexibility.
I think we can support this in Dynamo itself by being clever about the boostrap room
An idea: align bootstrap_room with Prefill DP Rank (doesnt require sglang changes)
Generate bootstrap_room values that encode the chosen prefill DP rank:
Flow would look like
Proposed Solution
.
Estimated PR Size
XS (1-10 lines)
Files/Components Affected
Would close DYN-1841