Use cudaRuntimeGetVersion for cudaMemcpyBatchAsync ABI dispatch

Kangyan-Zhou · claude · Kangyan-Zhou · commit 3d3428e4f2b0 · 2026-04-19T11:26:32.000-07:00
Addresses the review on #23172: #23172 (comment) cudaMemcpyBatchAsync is a libcudart (runtime) symbol; the ABI of the function dlsym'd into this process is owned by the libcudart that's actually loaded, not by the host's kernel driver. Dispatching on cudaDriverGetVersion() breaks in the common container case where a cu12 runtime is paired with a cu13-capable host driver: driver=13000 steers us to the 8-param v13 call, but the symbol resolves to v12 (9 params with failIdx), so the stream argument lands in a wrong slot and we segfault — the exact crash this fix was supposed to prevent. Reproduced on ion-user-9 with lmsysorg/sglang:dev (cu12.9 runtime): cudaDriverGetVersion() = 13000 cudaRuntimeGetVersion() = 12090 v12 dispatch of dlsym'd symbol: cudaSuccess, exit 0 v13 dispatch of dlsym'd symbol: Segmentation fault (core dumped) Switching the signature-selection to cudaRuntimeGetVersion makes the choice follow the loaded libcudart, which is what actually determines the ABI. The existing cudaDriverGetVersion guard above is kept — it remains the right knob for the capability check since cudaMemcpyBatch requires a 12.8+ driver regardless of the runtime version. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
diff --git a/sgl-kernel/csrc/kvcacheio/transfer.cu b/sgl-kernel/csrc/kvcacheio/transfer.cu
@@ -812,9 +812,19 @@ inline void transfer_kv_page_first_direct_impl(
     return;
   }
 
-  // CUDA 13.0 removed the failIdx parameter from cudaMemcpyBatchAsync.
-  // Use runtime version to select the correct signature for binary portability.
-  const bool use_v13_signature = driver_version >= 13000;
+  // CUDA 13.0 removed the failIdx parameter from cudaMemcpyBatchAsync. The ABI
+  // of the dlsym'd symbol is determined by the libcudart loaded in this process,
+  // not the host driver — a cu12 runtime on a cu13 driver host (common in
+  // containers) still exposes the 9-param v12 signature. Dispatching on the
+  // driver version here would segfault in that case (verified empirically).
+  // Use cudaRuntimeGetVersion so the signature follows the runtime.
+  int runtime_version = 0;
+  cudaError_t runtime_version_err = cudaRuntimeGetVersion(&runtime_version);
+  if (runtime_version_err != cudaSuccess) {
+    fallback_to_page_copy();
+    return;
+  }
+  const bool use_v13_signature = runtime_version >= 13000;
 
   size_t num_copies = 0;
   std::vector<void*> batch_srcs;
