Motivation

When prefill and decode use different kv_cache_dtype (e.g. prefill bf16, decode fp8_e4m3), the per-page item_len differs (32768 vs 16384). Since the transfer layer uses prefill's item_len to compute both src and dst addresses, this causes dst address overflow into unregistered memory at high concurrency, crashing with "Requested address not found" (Mooncake) or "NIXL_ERR_NOT_FOUND" (NIXL).

There is no dtype conversion in the transfer path, so mismatched dtype silently produces corrupted data at low concurrency and crashes at high concurrency. Add an item_len check when prefill receives decode's KV registration; on mismatch, log an error and refuse the peer instead of crashing the prefill process.

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.