llama : do not skip iGPU when only RPC devices are present (#23868)

rgerganov · web-flow · commit 1738129bee5c · 2026-05-30T07:48:22.000+03:00
After #23007 reclassified integrated CUDA/HIP devices as IGPU, the device selection logic dropped the local iGPU whenever any RPC server was added, because RPC devices made `model->devices` non-empty. On systems where the "iGPU" is the main compute device (e.g. Strix Halo with 128 GiB of unified memory), this caused all tensors to be allocated on the RPC peer alone and model loading to fail. Gate the iGPU inclusion on `gpus.empty()` instead, so RPC peers no longer suppress the local iGPU. closes: #23858
diff --git a/src/llama.cpp b/src/llama.cpp
@@ -239,8 +239,9 @@ static bool llama_prepare_model_devices(const llama_model_params & params, llama
         // add GPUs
         model->devices.insert(model->devices.end(), gpus.begin(), gpus.end());
 
-        // add integrated GPUs only if no other devices were found
-        if (model->devices.empty()) {
+        // add integrated GPUs only if no discrete GPUs were found
+        // (RPC servers do not count, otherwise the local iGPU would be dropped on iGPU+RPC setups)
+        if (gpus.empty()) {
             model->devices.insert(model->devices.end(), igpus.begin(), igpus.end());
         }
     }

Original file line number	Diff line number	Diff line change
`@@ -239,8 +239,9 @@ static bool llama_prepare_model_devices(const llama_model_params & params, llama`
`239`	`239`	`// add GPUs`
`240`	`240`	`model->devices.insert(model->devices.end(), gpus.begin(), gpus.end());`
`241`	`241`
`242`		`- // add integrated GPUs only if no other devices were found`
`243`		`- if (model->devices.empty()) {`
	`242`	`+ // add integrated GPUs only if no discrete GPUs were found`
	`243`	`+ // (RPC servers do not count, otherwise the local iGPU would be dropped on iGPU+RPC setups)`
	`244`	`+ if (gpus.empty()) {`
`244`	`245`	`model->devices.insert(model->devices.end(), igpus.begin(), igpus.end());`
`245`	`246`	`}`
`246`	`247`	`}`