Speculative Decoding Development Roadmap (2026 Q2)

• adaptive speculative decoding [Roadmap] Adaptive Speculative Decoding Roadmap #23705
• spec v2 [Feature] Overlap Spec Support #11762
  • enable v2 by default Use spec v2 by default #21062
  • fully overlap (remove wait_for_verify sync)
    • initial pr [wip][perf] fully overlap spec v2, remove wait_for_verify sync #23452
    • avoid using cpu metadata in attention backends
  • constrained decoding feat: Full Support for Overlapped Constrained Decoding + Spec V2 #15623
  • topk > 1 (optional)
• piecewise-cuda-graph
  • make spec decoding compatible with piecewise-cuda-graph Allow piecewise CUDA graph with speculative decoding #22128
• dflash @dcw02
  • initial pr [Feature] Add DFLASH speculative decoding support #22077
  • support spec v2 [Feature] Spec V2 DFlash Support #23000
  • ... (todo)
• refactor
  • rename some ambiguous variable names and concepts @Qiaolin-Yu
  • Roadmap: Clean up spec info / MIMO hidden states handling #21058
• parallel spec decoding
  • initial pr [Feature]SPECTRE: Parallel Speculative Decoding with a Multi-Tenant Remote Drafter #22272
• ngram [Roadmap] Further Ngram Speculative Decoding Support #21052 @kpham-sgl