Motivation

Prefill-Decode Disaggregation has been an important and widely adopted feature for LLM inference production.
Although we have almost made it compatible with different model architectures, all parallel strategies, and almost all advanced features, there are always new models/features coming up, which we also want to support in PD mode.

This issue will act as a roadmap or a tracker for those who are interested in the latest progress of this feature (at least it will include some related experimental features we will conduct in 2026 Q2).

Interface refactoring

• Support query dp rank from bootstrap server, which allows us to achieve better DP load balance [PD-Disagg] Support query dp rank from bootstrap server. #19168
• Refactor PD decode KV receiver bootstrap for better readability and bugfix [Feature] Refactor PD decode KV receiver bootstrap for synchronized queue state #21680 [Disagg DP-Balancing] Refactor Disagg Conn and Fix Hang with total_request/total_tokens Balancing #21299 @weireweire

Advanced feature and usage

• Context parallel compatibility after refactor [PD] Support PD with context parallel after refactor #19504 [PD] Enable all CP ranks for KVCache transfer  #19765
• GPU staging buffer for accelerating heterogeneous TP KV transfer [Disagg] GPU staging buffer with dynamic ring allocator for heterogeneous TP KV transfer #19890 @YAMY1234
  • For MHA/GQA models, there are many small pieces of KVCache transfer for a heterogeneous TP setup of PD, which will slow down the transfer process. This feature will gather the KV heads at prefill, then send the kvcache to decode by uniting them as a large piece through a ring-based staging buffer (really helpful for GB platforms).
• Compatibility with Hisparse [PD]: Add support for HiSparse to directly transfer the cache from Prefill to Decode DRAM. #21591 @hzh0425
  • This feature will allow us to send kvcache from prefill HBM to decode host memory when enabling hisparse for NSA models (DeepSeek V3.2/GLM 5). It will help us to get a much bigger batch size on the decode side since we only keep the topk sparse KVCache in HBM, and make all other KVCache stored in the host memory.
• Prefill send delta KVCache while Decode fetch prefix from Hicache for agentic use cases @hzh0425
• Decode RadixCache [P/D disagg] - support decode side radix cache #19746 @ishandhanani
  • For agentic scenarios, decode can now use radix cache to reuse shared prefixes and request only the delta KV from prefill instead of transferring the full prefix on every turn.
• Better compatibility with Spec decode: support heterogeneous target/draft model architectures [feature][PD disaggregation] KV cache transfer for speculative decoding with different target/draft model architectures (MLA and MHA combinations) #20698
  • This feature will allow us to use different draft architectures from target models when enabling speculative decoding or MTP (specforge/torchspec might need it).
• Layerwise KVCache transfer [Draft] feat: Mooncake support layerwise kv cache transfer #19931 @zhangxiaolei123456 @UNIDY2002 [Disagg] Layer-pipelined KV transfer: overlap RDMA with GPU compute #23515
• Support Prefill PP + MTP for PD Disaggregation mode [RFC] GLM-5 Spec. Decode + PP in Disaggregation Decode Mode #23162

Routing strategy enhancement for agentic serving

• Support heterogeneous instances in the same (Prefill/Decode) pool while simultaneously enabling seamless pairing with heterogeneous (Decode/Prefill) instances. (We have supported this from the very beginning.)
• Support extended roles in RBG, (long context prefill instance, short context prefill instance, long context decode instance, short context decode instance) @cheyang Do we support this now?
• Support session-based load-balanced-aware routing strategy in sglang router @whybeyoung

New model support:

• Qwen3.5: hybrid linear attention support PP+PD [Fix] Fix Qwen3.5 MoE model loading and Mamba cache sharding in PP mode #21448 feat(Qwen3.5): hybrid linear attention support PP+PD #19254 @zhangxiaolei123456 @sufeng-buaa

More will be added. Feel free to comment any feature requests down here.