Skip to content

[CONTRIBUTION]: Support variable DP rank + P/D routing in Slgnag #5638

@ishandhanani

Description

@ishandhanani

Type of Change

Bug fix

Problem Statement

##Summary

When using disaggregated prefill/decode with DP attention mode, the current implementation couples the decode routing decision with the KV transfer source identification. This prevents independent DP size configurations between prefill and decode servers and limits routing flexibility.

I think we can support this in Dynamo itself by being clever about the boostrap room

An idea: align bootstrap_room with Prefill DP Rank (doesnt require sglang changes)

Generate bootstrap_room values that encode the chosen prefill DP rank:

// In prefill_router.rs

// Current (sequential):
// let bootstrap_room = BOOTSTRAP_ROOM_COUNTER.fetch_add(1, Ordering::Relaxed);

// Proposed (encodes prefill dp_rank):
let prefill_dp_size = get_prefill_dp_size();  // From worker registration
let base = BOOTSTRAP_ROOM_COUNTER.fetch_add(prefill_dp_size, Ordering::Relaxed);
let bootstrap_room = base + chosen_prefill_dp_rank;

Flow would look like

Dynamo KV router picks prefill DP rank 2
        │
        ▼
bootstrap_room = base + 2 (guarantees bootstrap_room % dp_size == 2)
        │
        ├──► Prefill routing: Dynamo routes to DP rank 2
        │
        ▼
Decode request (no dp_rank set)
        │
        ├──► Decode routing: Dynamo KV router picks independently (e.g., DP rank 0)
        │
        └──► KV receiver: prefill_dp_rank = bootstrap_room % prefill_dp_size = 2 ✓

Proposed Solution

.

Estimated PR Size

XS (1-10 lines)

Files/Components Affected

Would close DYN-1841

Metadata

Metadata

Assignees

No one assigned

    Labels

    contribution-requestExternal contributor proposing to implement a change

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions