Description
I can use NVIDIA GPUs from normal Docker containers with --gpus all, but nomeclaw onbording with --gpu fails after the sandbox container is created with GPU access.
Reproduction Steps
Docker and nvidia-ctk are configurated correctly.
$ cat /etc/docker/daemon.json
{
"default-runtime": "nvidia",
"runtimes": {
"nvidia": {
"args": [],
"path": "nvidia-container-runtime"
}
},
"features": {
"cdi": true
}
}
$ sudo systemctl restart docker
$ sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
INFO[0000] Using /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.565.77
WARN[0000] Ignoring error in locating libnvidia-sandboxutils.so.1: pattern libnvidia-sandboxutils.so.1 not found
libnvidia-sandboxutils.so.1: not found
WARN[0000] Failed to init nvsandboxutils: ERROR_LIBRARY_LOAD; ignoring
INFO[0000] Auto-detected mode as 'nvml'
INFO[0000] Selecting /dev/nvidia0 as /dev/nvidia0
INFO[0000] Selecting /dev/dri/card1 as /dev/dri/card1
WARN[0000] Could not locate /dev/dri/controlD65: pattern /dev/dri/controlD65 not found
INFO[0000] Selecting /dev/dri/renderD128 as /dev/dri/renderD128
INFO[0000] Selecting /dev/nvidia0 as /dev/nvidia0
INFO[0000] Selecting /dev/dri/card1 as /dev/dri/card1
WARN[0000] Could not locate /dev/dri/controlD65: pattern /dev/dri/controlD65 not found
INFO[0000] Selecting /dev/dri/renderD128 as /dev/dri/renderD128
INFO[0000] Using driver version 565.77
INFO[0000] Selecting /dev/nvidia-modeset as /dev/nvidia-modeset
INFO[0000] Selecting /dev/nvidia-uvm-tools as /dev/nvidia-uvm-tools
INFO[0000] Selecting /dev/nvidia-uvm as /dev/nvidia-uvm
INFO[0000] Selecting /dev/nvidiactl as /dev/nvidiactl
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1.1.2 as /usr/lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1.1.2
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-egl-wayland.so.1.1.17 as /usr/lib/x86_64-linux-gnu/libnvidia-egl-wayland.so.1.1.17
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.565.77
WARN[0000] Could not locate libnvidia-vulkan-producer.so.565.77: pattern libnvidia-vulkan-producer.so.565.77 not found
libnvidia-vulkan-producer.so.565.77: not found
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/nvidia/xorg/nvidia_drv.so as /usr/lib/x86_64-linux-gnu/nvidia/xorg/nvidia_drv.so
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so.565.77 as /usr/lib/x86_64-linux-gnu/nvidia/xorg/libglxserver_nvidia.so.565.77
INFO[0000] Selecting /usr/share/glvnd/egl_vendor.d/10_nvidia.json as /usr/share/glvnd/egl_vendor.d/10_nvidia.json
INFO[0000] Selecting /usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json as /usr/share/egl/egl_external_platform.d/15_nvidia_gbm.json
INFO[0000] Selecting /usr/share/egl/egl_external_platform.d/10_nvidia_wayland.json as /usr/share/egl/egl_external_platform.d/10_nvidia_wayland.json
INFO[0000] Selecting /usr/share/nvidia/nvoptix.bin as /usr/share/nvidia/nvoptix.bin
INFO[0000] Selecting /usr/share/X11/xorg.conf.d/10-nvidia.conf as /usr/share/X11/xorg.conf.d/10-nvidia.conf
WARN[0000] Could not locate X11/xorg.conf.d/nvidia-drm-outputclass.conf: pattern X11/xorg.conf.d/nvidia-drm-outputclass.conf not found
INFO[0000] Selecting /usr/share/vulkan/icd.d/nvidia_icd.json as /etc/vulkan/icd.d/nvidia_icd.json
WARN[0000] Could not locate vulkan/icd.d/nvidia_layers.json: pattern vulkan/icd.d/nvidia_layers.json not found
pattern vulkan/icd.d/nvidia_layers.json not found
INFO[0000] Selecting /usr/share/vulkan/implicit_layer.d/nvidia_layers.json as /etc/vulkan/implicit_layer.d/nvidia_layers.json
WARN[0000] Could not locate vulkan/icd.d/nvidia_icd.x86_64.json: pattern vulkan/icd.d/nvidia_icd.x86_64.json not found
pattern vulkan/icd.d/nvidia_icd.x86_64.json not found
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.565.77 as /usr/lib/x86_64-linux-gnu/libEGL_nvidia.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.565.77 as /usr/lib/x86_64-linux-gnu/libGLESv1_CM_nvidia.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.565.77 as /usr/lib/x86_64-linux-gnu/libGLESv2_nvidia.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.565.77 as /usr/lib/x86_64-linux-gnu/libGLX_nvidia.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libcuda.so.565.77 as /usr/lib/x86_64-linux-gnu/libcuda.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libcudadebugger.so.565.77 as /usr/lib/x86_64-linux-gnu/libcudadebugger.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvcuvid.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvcuvid.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-allocator.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-cfg.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-eglcore.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-encode.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-fbc.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-glcore.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-glsi.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-glvkspirv.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-gpucomp.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-gpucomp.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-ngx.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-ngx.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-nvvm.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-nvvm.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-opticalflow.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-pkcs11-openssl3.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-pkcs11-openssl3.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-pkcs11.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-pkcs11.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-rtcore.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvidia-vksc-core.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvidia-vksc-core.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/libnvoptix.so.565.77 as /usr/lib/x86_64-linux-gnu/libnvoptix.so.565.77
INFO[0000] Selecting /usr/lib/x86_64-linux-gnu/vdpau/libvdpau_nvidia.so.565.77 as /usr/lib/x86_64-linux-gnu/vdpau/libvdpau_nvidia.so.565.77
INFO[0000] Selecting /run/nvidia-persistenced/socket as /run/nvidia-persistenced/socket
WARN[0000] Could not locate /nvidia-fabricmanager/socket: pattern /nvidia-fabricmanager/socket not found
WARN[0000] Could not locate /tmp/nvidia-mps: pattern /tmp/nvidia-mps not found
INFO[0000] Selecting /lib/firmware/nvidia/565.77/gsp_ga10x.bin as /lib/firmware/nvidia/565.77/gsp_ga10x.bin
INFO[0000] Selecting /lib/firmware/nvidia/565.77/gsp_tu10x.bin as /lib/firmware/nvidia/565.77/gsp_tu10x.bin
INFO[0000] Selecting /usr/bin/nvidia-smi as /usr/bin/nvidia-smi
INFO[0000] Selecting /usr/bin/nvidia-debugdump as /usr/bin/nvidia-debugdump
INFO[0000] Selecting /usr/bin/nvidia-persistenced as /usr/bin/nvidia-persistenced
INFO[0000] Selecting /usr/bin/nvidia-cuda-mps-control as /usr/bin/nvidia-cuda-mps-control
INFO[0000] Selecting /usr/bin/nvidia-cuda-mps-server as /usr/bin/nvidia-cuda-mps-server
WARN[0000] Could not locate nvidia-imex: pattern nvidia-imex not found
WARN[0000] Could not locate nvidia-imex-ctl: pattern nvidia-imex-ctl not found
INFO[0000] Generated CDI spec with version 1.0.0
$ sudo chmod 644 /etc/cdi/nvidia.yaml
$ nvidia-ctk cdi list
INFO[0000] Found 3 CDI devices
nvidia.com/gpu=0
nvidia.com/gpu=GPU-f4737b8e-40dc-1a26-f9ec-a2cae2ba46a9
nvidia.com/gpu=all
$ docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
Wed May 27 09:27:05 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.77 Driver Version: 565.77 CUDA Version: 12.7 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 4090 Off | 00000000:01:00.0 Off | 0 |
| 0% 52C P8 21W / 500W | 449MiB / 23028MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+
However, nemoclaw omboaring is failed as below.
$ git clone https://github.com/NVIDIA/NemoClaw.git
$ cd NemoClaw/
$ git branch --show-current
main
$ nvm use 22
Now using node v22.22.3 (npm v10.9.8)
$ npm ci
$ npm install --include=dev --ignore-scripts
$ npm run build:cli
$ node ./bin/nemoclaw.js --version
nemoclaw v0.0.52-18-gd31214441
$ node ./bin/nemoclaw.js onboard \
--fresh \
--gpu \
--control-ui-port 18790
NemoClaw Onboarding
===================
[1/8] Preflight checks
──────────────────────────────────────────────────
✓ Docker is running
⚠ Container DNS probe inconclusive (reason: no_output).
docker run produced no output (timed out or failed to start)
Proceeding. If the sandbox build later hangs at `npm ci`, see issue #2101.
✓ Container runtime: docker
✓ Container runtime resources: 24 vCPU / 94.0 GiB
✓ openshell CLI: openshell 0.0.44
✓ Port 8080 available (OpenShell gateway)
✓ Port 18790 available (NemoClaw dashboard)
✓ NVIDIA GPU detected (NVIDIA GeForce RTX 4090, 23028 MB)
✓ Docker CDI GPU support detected (/etc/cdi/nvidia.yaml, /var/run/cdi/nvidia.yaml)
✓ Sandbox GPU: enabled (1)
✓ Memory OK: 96292 MB RAM + 2047 MB swap
GPU passthrough requested; passing --gpu to OpenShell gateway and sandbox creation.
[2/8] Starting OpenShell gateway
──────────────────────────────────────────────────
Starting OpenShell Docker-driver gateway...
Gateway log: /home/masaki/.local/state/nemoclaw/openshell-docker-gateway/openshell-gateway.log
OpenShell gateway compatibility patch active (host glibc 2.35 is older than openshell-gateway requirement 2.39).
Running openshell-gateway inside a Docker compatibility container.
Compatibility gateway bind: 0.0.0.0 (required for Docker sandbox callbacks).
✓ Docker-driver gateway is healthy
[3/8] Configuring inference provider
──────────────────────────────────────────────────
Detected local inference option: Ollama
Inference options:
1) NVIDIA Endpoints
2) OpenAI
3) Other OpenAI-compatible endpoint
4) Anthropic
5) Other Anthropic-compatible endpoint
6) Google Gemini
7) Local Ollama (localhost:11434) — running (suggested)
8) Model Router (experimental)
Choose [1]: 7
Configuring Ollama systemd loopback override...
Applying an Ollama systemd override (OLLAMA_HOST=127.0.0.1:11434). The next steps use sudo to write the drop-in, reload systemd, and restart the service; you may be prompted for your password.
✓ Using Ollama on localhost:11434 (proxy on :11435)
Ollama models:
1) gemma4:31b
2) bge-m3:latest
3) gpt-oss:20b
4) Other...
Choose model [1]: 3
Loading Ollama model: gpt-oss:20b
Chat Completions API available — OpenClaw will use openai-completions.
✓ Using Ollama runtime context length: 4096 tokens
Sandbox name (1-63 characters, lowercase, starts with a letter, letters/numbers/internal hyphens only, ends with letter/number) [my-assistant]: my-assistant
──────────────────────────────────────────────────
Review configuration
──────────────────────────────────────────────────
Provider: ollama-local
Model: gpt-oss:20b
API key: (not required for ollama-local)
Web search: disabled
Managed tools: none
Messaging: none
Sandbox name: my-assistant
Note: Sandbox build typically takes 3–8 minutes on this host.
──────────────────────────────────────────────────
Web search and messaging channels will be prompted next.
Apply this configuration? [Y/n]: Y
[4/8] Setting up inference provider
──────────────────────────────────────────────────
✓ Active gateway set to 'nemoclaw'
✓ Created provider ollama-local
Gateway inference configured:
Route: inference.local
Provider: ollama-local
Model: gpt-oss:20b
Version: 1
Timeout: 180s
Priming Ollama model: gpt-oss:20b
✓ Inference route set: ollama-local / gpt-oss:20b
Enable Brave Web Search? [y/N]: N
[5/8] Messaging channels
──────────────────────────────────────────────────
Available messaging channels:
[1] ○ telegram — Telegram bot messaging
[2] ○ discord — Discord bot messaging
[3] ○ wechat — WeChat (personal) bot messaging
[4] ○ slack — Slack bot messaging
[5] ○ whatsapp — WhatsApp Web messaging (QR pairing)
Press 1-5 to toggle, Enter when done:
Skipping messaging channels.
Resource profiles:
1) creator (cpu=50%, ram=50%)
2) gamer (cpu=25%, ram=25%)
3) game-developer (cpu=60%, ram=60%)
4) developer (cpu=75%, ram=75%)
5) custom (enter values manually)
6) No profile (OpenShell defaults)
Choose [6]: 6
[6/8] Creating sandbox
──────────────────────────────────────────────────
Docker-driver GPU patch active; allowing /proc writes required by Docker GPU initialization.
Creating sandbox 'my-assistant' (this takes a few minutes on first run)...
Pinning base image to sha256:62abb74f274f...
Building sandbox image...
Building image openshell/sandbox-from:1779882126 from /tmp/nemoclaw-build-dISX6F/Dockerfile
Step 1/79 : ARG BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base@sha256:62abb74f274fcb4ee1a2b96f57f437ea581937b4...
Step 1/79 completed in 0.0s (ARG BASE_IMAGE=ghcr.io/nvidia/nemoclaw/sandbox-base@sha256:62abb74f274fcb4ee1a2b96...
Step 2/79 : FROM node:22-trixie-slim@sha256:2d9f5c76c8f4dd36e8f253bee5d828a83a6c09f36188f0b0414325232e0b175d AS...
Step 2/79 completed in 0.0s (FROM node:22-trixie-slim@sha256:2d9f5c76c8f4dd36e8f253bee5d828a83a6c09f36188f0b041...
Step 3/79 : ENV NPM_CONFIG_AUDIT=false NPM_CONFIG_FUND=false NPM_CONFIG_UPDATE_NOTIFIER=false NPM_C...
Step 3/79 completed in 0.0s (ENV NPM_CONFIG_AUDIT=false NPM_CONFIG_FUND=false NPM_CONFIG_UPDATE_NOTIFIER=false ...
Step 4/79 : COPY nemoclaw/package.json nemoclaw/package-lock.json nemoclaw/tsconfig.json /opt/nemoclaw/
Step 4/79 completed in 0.0s (COPY nemoclaw/package.json nemoclaw/package-lock.json nemoclaw/tsconfig.json /opt/...
Step 5/79 : COPY nemoclaw/src/ /opt/nemoclaw/src/
Step 5/79 completed in 0.0s (COPY nemoclaw/src/ /opt/nemoclaw/src/)
Step 6/79 : WORKDIR /opt/nemoclaw
Step 6/79 completed in 0.0s (WORKDIR /opt/nemoclaw)
Step 7/79 : RUN npm ci && npm run build
Step 7/79 completed in 0.0s (RUN npm ci && npm run build)
Step 8/79 : FROM ${BASE_IMAGE}
Step 8/79 completed in 0.0s (FROM ${BASE_IMAGE})
Step 9/79 : ARG OPENCLAW_VERSION=2026.5.22
Step 9/79 completed in 0.0s (ARG OPENCLAW_VERSION=2026.5.22)
Step 10/79 : ARG OPENCLAW_2026_5_22_INTEGRITY=sha512-m+zgBELGbCHjWB1IWF5WSWNPr480cMKOMff2OF72c8A0AMD4hC/9+qwYtz...
Step 10/79 completed in 0.0s (ARG OPENCLAW_2026_5_22_INTEGRITY=sha512-m+zgBELGbCHjWB1IWF5WSWNPr480cMKOMff2OF72c...
Step 11/79 : RUN set -eu; apt-mark manual procps e2fsprogs 2>/dev/null || true; (apt-get remove --purge...
Step 11/79 completed in 0.0s (RUN set -eu; apt-mark manual procps e2fsprogs 2>/dev/null || true; (apt-get remov...
Step 12/79 : COPY --from=builder /opt/nemoclaw/dist/ /opt/nemoclaw/dist/
Step 12/79 completed in 0.0s (COPY --from=builder /opt/nemoclaw/dist/ /opt/nemoclaw/dist/)
Step 13/79 : COPY nemoclaw/openclaw.plugin.json /opt/nemoclaw/
Step 13/79 completed in 0.0s (COPY nemoclaw/openclaw.plugin.json /opt/nemoclaw/)
Step 14/79 : COPY nemoclaw/package.json nemoclaw/package-lock.json /opt/nemoclaw/
Step 14/79 completed in 0.0s (COPY nemoclaw/package.json nemoclaw/package-lock.json /opt/nemoclaw/)
Step 15/79 : COPY nemoclaw-blueprint/ /opt/nemoclaw-blueprint/
Step 15/79 completed in 0.0s (COPY nemoclaw-blueprint/ /opt/nemoclaw-blueprint/)
Step 16/79 : RUN chmod -R a+rX /opt/nemoclaw /opt/nemoclaw-blueprint/
Step 16/79 completed in 0.0s (RUN chmod -R a+rX /opt/nemoclaw /opt/nemoclaw-blueprint/)
Step 17/79 : WORKDIR /opt/nemoclaw
Step 17/79 completed in 0.0s (WORKDIR /opt/nemoclaw)
Step 18/79 : ENV NPM_CONFIG_AUDIT=false NPM_CONFIG_FUND=false NPM_CONFIG_UPDATE_NOTIFIER=false NPM_...
Step 18/79 completed in 0.0s (ENV NPM_CONFIG_AUDIT=false NPM_CONFIG_FUND=false NPM_CONFIG_UPDATE_NOTIFIER=false...
Step 19/79 : RUN npm ci --omit=dev
Step 19/79 completed in 0.0s (RUN npm ci --omit=dev)
Step 20/79 : COPY scripts/patch-openclaw-tool-catalog.js /usr/local/lib/nemoclaw/patch-openclaw-tool-catalog.js
Step 20/79 completed in 0.0s (COPY scripts/patch-openclaw-tool-catalog.js /usr/local/lib/nemoclaw/patch-opencla...
Step 21/79 : COPY scripts/patch-openclaw-chat-send.js /usr/local/lib/nemoclaw/patch-openclaw-chat-send.js
Step 21/79 completed in 0.0s (COPY scripts/patch-openclaw-chat-send.js /usr/local/lib/nemoclaw/patch-openclaw-c...
Step 22/79 : RUN chmod 755 /usr/local/lib/nemoclaw/patch-openclaw-tool-catalog.js /usr/local/lib/nemocl...
Step 22/79 completed in 0.0s (RUN chmod 755 /usr/local/lib/nemoclaw/patch-openclaw-tool-catalog.js /usr/local/l...
Step 23/79 : RUN set -eu; echo "$OPENCLAW_VERSION" | grep -qxE '[0-9]+(\.[0-9]+)*' || { echo "ERROR...
Step 23/79 completed in 0.0s (RUN set -eu; echo "$OPENCLAW_VERSION" | grep -qxE '[0-9]+(\.[0-9]+)*' || { echo "...
Step 24/79 : RUN set -eu; OC_DIST=/usr/local/lib/node_modules/openclaw/dist; OC_VERSION="$(openclaw --v...
Step 24/79 completed in 0.0s (RUN set -eu; OC_DIST=/usr/local/lib/node_modules/openclaw/dist; OC_VERSION="$(ope...
Step 25/79 : RUN node /usr/local/lib/nemoclaw/patch-openclaw-chat-send.js /usr/local/lib/node_modules/openc...
Step 25/79 completed in 0.0s (RUN node /usr/local/lib/nemoclaw/patch-openclaw-chat-send.js /usr/local/lib/node_...
Step 26/79 : RUN node /usr/local/lib/nemoclaw/patch-openclaw-tool-catalog.js /usr/local/lib/node_modules/op...
Step 26/79 completed in 0.0s (RUN node /usr/local/lib/nemoclaw/patch-openclaw-tool-catalog.js /usr/local/lib/no...
Step 27/79 : RUN mkdir -p /sandbox/.nemoclaw/blueprints/0.1.0 && cp -r /opt/nemoclaw-blueprint/* /sandbox/....
Step 27/79 completed in 0.0s (RUN mkdir -p /sandbox/.nemoclaw/blueprints/0.1.0 && cp -r /opt/nemoclaw-blueprint...
Step 28/79 : COPY scripts/lib/sandbox-init.sh /usr/local/lib/nemoclaw/sandbox-init.sh
Step 28/79 completed in 0.0s (COPY scripts/lib/sandbox-init.sh /usr/local/lib/nemoclaw/sandbox-init.sh)
Step 29/79 : COPY scripts/nemoclaw-start.sh /usr/local/bin/nemoclaw-start
Step 29/79 completed in 0.0s (COPY scripts/nemoclaw-start.sh /usr/local/bin/nemoclaw-start)
Step 30/79 : COPY nemoclaw-blueprint/scripts/*.js /usr/local/lib/nemoclaw/preloads/
Step 30/79 completed in 0.0s (COPY nemoclaw-blueprint/scripts/*.js /usr/local/lib/nemoclaw/preloads/)
Step 31/79 : COPY scripts/codex-acp-wrapper.sh /usr/local/bin/nemoclaw-codex-acp
Step 31/79 completed in 0.0s (COPY scripts/codex-acp-wrapper.sh /usr/local/bin/nemoclaw-codex-acp)
Step 32/79 : COPY scripts/generate-openclaw-config.py /usr/local/lib/nemoclaw/generate-openclaw-config.py
Step 32/79 completed in 0.0s (COPY scripts/generate-openclaw-config.py /usr/local/lib/nemoclaw/generate-opencla...
Step 33/79 : COPY scripts/seed-wechat-accounts.py /usr/local/lib/nemoclaw/seed-wechat-accounts.py
Step 33/79 completed in 0.0s (COPY scripts/seed-wechat-accounts.py /usr/local/lib/nemoclaw/seed-wechat-accounts...
Step 34/79 : COPY nemoclaw-blueprint/openclaw-plugins/ /usr/local/share/nemoclaw/openclaw-plugins/
Step 34/79 completed in 0.0s (COPY nemoclaw-blueprint/openclaw-plugins/ /usr/local/share/nemoclaw/openclaw-plug...
Step 35/79 : RUN chmod 755 /usr/local/bin/nemoclaw-start /usr/local/bin/nemoclaw-codex-acp /usr/local/l...
Step 35/79 completed in 0.0s (RUN chmod 755 /usr/local/bin/nemoclaw-start /usr/local/bin/nemoclaw-codex-acp /us...
Step 36/79 : ARG NEMOCLAW_MODEL=gpt-oss:20b
Step 36/79 completed in 0.0s (ARG NEMOCLAW_MODEL=gpt-oss:20b)
Step 37/79 : ARG NEMOCLAW_PROVIDER_KEY=inference
Step 37/79 completed in 0.0s (ARG NEMOCLAW_PROVIDER_KEY=inference)
Step 38/79 : ARG NEMOCLAW_PRIMARY_MODEL_REF=inference/gpt-oss:20b
Step 38/79 completed in 0.0s (ARG NEMOCLAW_PRIMARY_MODEL_REF=inference/gpt-oss:20b)
Step 39/79 : ARG CHAT_UI_URL=http://127.0.0.1:18790
Step 39/79 completed in 0.0s (ARG CHAT_UI_URL=http://127.0.0.1:18790)
Step 40/79 : ARG NEMOCLAW_INFERENCE_BASE_URL=https://inference.local/v1
Step 40/79 completed in 0.0s (ARG NEMOCLAW_INFERENCE_BASE_URL=https://inference.local/v1)
Step 41/79 : ARG NEMOCLAW_INFERENCE_API=openai-completions
Step 41/79 completed in 0.0s (ARG NEMOCLAW_INFERENCE_API=openai-completions)
Step 42/79 : ARG NEMOCLAW_CONTEXT_WINDOW=4096
Step 42/79 completed in 0.0s (ARG NEMOCLAW_CONTEXT_WINDOW=4096)
Step 43/79 : ARG NEMOCLAW_MAX_TOKENS=4096
Step 43/79 completed in 0.0s (ARG NEMOCLAW_MAX_TOKENS=4096)
Step 44/79 : ARG NEMOCLAW_REASONING=false
Step 44/79 completed in 0.0s (ARG NEMOCLAW_REASONING=false)
Step 45/79 : ARG NEMOCLAW_INFERENCE_INPUTS=text
Step 45/79 completed in 0.0s (ARG NEMOCLAW_INFERENCE_INPUTS=text)
Step 46/79 : ARG NEMOCLAW_AGENT_TIMEOUT=600
Step 46/79 completed in 0.0s (ARG NEMOCLAW_AGENT_TIMEOUT=600)
Step 47/79 : ARG NEMOCLAW_AGENT_HEARTBEAT_EVERY=
Step 47/79 completed in 0.0s (ARG NEMOCLAW_AGENT_HEARTBEAT_EVERY=)
Step 48/79 : ARG NEMOCLAW_INFERENCE_COMPAT_B64=e30=
Step 48/79 completed in 0.0s (ARG NEMOCLAW_INFERENCE_COMPAT_B64=e30=)
Step 49/79 : ARG NEMOCLAW_MESSAGING_CHANNELS_B64=W10=
Step 49/79 completed in 0.0s (ARG NEMOCLAW_MESSAGING_CHANNELS_B64=W10=)
Step 50/79 : ARG NEMOCLAW_MESSAGING_ALLOWED_IDS_B64=e30=
Step 50/79 completed in 0.0s (ARG NEMOCLAW_MESSAGING_ALLOWED_IDS_B64=e30=)
Step 51/79 : ARG NEMOCLAW_DISCORD_GUILDS_B64=e30=
Step 51/79 completed in 0.0s (ARG NEMOCLAW_DISCORD_GUILDS_B64=e30=)
Step 52/79 : ARG NEMOCLAW_TELEGRAM_CONFIG_B64=e30=
Step 52/79 completed in 0.0s (ARG NEMOCLAW_TELEGRAM_CONFIG_B64=e30=)
Step 53/79 : ARG NEMOCLAW_WECHAT_CONFIG_B64=e30=
Step 53/79 completed in 0.0s (ARG NEMOCLAW_WECHAT_CONFIG_B64=e30=)
Step 54/79 : ARG NEMOCLAW_SLACK_CONFIG_B64=e30=
Step 54/79 completed in 0.0s (ARG NEMOCLAW_SLACK_CONFIG_B64=e30=)
Step 55/79 : ARG NEMOCLAW_DISABLE_DEVICE_AUTH=1
Step 55/79 completed in 0.0s (ARG NEMOCLAW_DISABLE_DEVICE_AUTH=1)
Step 56/79 : ARG NEMOCLAW_BUILD_ID=1779882126254
Step 56/79 completed in 0.1s (ARG NEMOCLAW_BUILD_ID=1779882126254)
Step 57/79 : ARG NEMOCLAW_DARWIN_VM_COMPAT=0
Step 57/79 completed in 0.1s (ARG NEMOCLAW_DARWIN_VM_COMPAT=0)
Step 58/79 : ARG NEMOCLAW_PROXY_HOST=10.200.0.1
Step 58/79 completed in 0.1s (ARG NEMOCLAW_PROXY_HOST=10.200.0.1)
Step 59/79 : ARG NEMOCLAW_PROXY_PORT=3128
Step 59/79 completed in 0.1s (ARG NEMOCLAW_PROXY_PORT=3128)
Step 60/79 : ARG NEMOCLAW_WEB_SEARCH_ENABLED=0
Step 60/79 completed in 0.1s (ARG NEMOCLAW_WEB_SEARCH_ENABLED=0)
Step 61/79 : ENV NEMOCLAW_MODEL=${NEMOCLAW_MODEL} NEMOCLAW_PROVIDER_KEY=${NEMOCLAW_PROVIDER_KEY} NEMOCL...
Step 61/79 completed in 0.1s (ENV NEMOCLAW_MODEL=${NEMOCLAW_MODEL} NEMOCLAW_PROVIDER_KEY=${NEMOCLAW_PROVIDER_KE...
Step 62/79 : WORKDIR /sandbox
Step 62/79 completed in 0.1s (WORKDIR /sandbox)
Step 63/79 : USER sandbox
Step 63/79 completed in 0.1s (USER sandbox)
Step 64/79 : RUN NEMOCLAW_OPENCLAW_MANAGED_PROXY=0 python3 /usr/local/lib/nemoclaw/generate-openclaw-config.py
Step 64/79 completed in 0.4s (RUN NEMOCLAW_OPENCLAW_MANAGED_PROXY=0 python3 /usr/local/lib/nemoclaw/generate-op...
Step 65/79 : RUN openclaw doctor --fix --non-interactive
Still building sandbox image... (20s elapsed)
Step 65/79 completed in 25.2s (RUN openclaw doctor --fix --non-interactive)
Step 66/79 : ENV NPM_CONFIG_OFFLINE=true NPM_CONFIG_AUDIT=false NPM_CONFIG_FUND=false
Step 66/79 completed in 0.1s (ENV NPM_CONFIG_OFFLINE=true NPM_CONFIG_AUDIT=false NPM_CONFIG_FUND=false)
Step 67/79 : RUN openclaw plugins install /opt/nemoclaw && openclaw plugins enable nemoclaw && openclaw...
Step 67/79 completed in 6.1s (RUN openclaw plugins install /opt/nemoclaw && openclaw plugins enable nemoclaw &&...
Step 68/79 : RUN python3 -c "import json, os; path = os.path.expanduser('~/.openclaw/openclaw.json'); cfg = jso...
Step 68/79 completed in 0.4s (RUN python3 -c "import json, os; path = os.path.expanduser('~/.openclaw/openclaw....
Step 69/79 : USER root
Step 69/79 completed in 0.1s (USER root)
Step 70/79 : RUN set -eu; config_dir=/sandbox/.openclaw; data_dir=/sandbox/.openclaw-data; legacy_l...
Step 70/79 completed in 9.1s (RUN set -eu; config_dir=/sandbox/.openclaw; data_dir=/sandbox/.openclaw-data; leg...
Step 71/79 : RUN if id gateway >/dev/null 2>&1 && id sandbox >/dev/null 2>&1; then if ! id -nG gateway ...
Step 71/79 completed in 0.4s (RUN if id gateway >/dev/null 2>&1 && id sandbox >/dev/null 2>&1; then if ! id -nG...
Step 72/79 : RUN set -eu; if [ -e /tmp/nemoclaw-legacy-openclaw-layout ]; then chown -R sandbox:san...
Step 72/79 completed in 0.4s (RUN set -eu; if [ -e /tmp/nemoclaw-legacy-openclaw-layout ]; then chown -R sandbo...
Step 73/79 : RUN if ! grep -q "/tmp/nemoclaw-proxy-env.sh" /etc/profile.d/nemoclaw-proxy.sh 2>/dev/null; then ...
Step 73/79 completed in 0.4s (RUN if ! grep -q "/tmp/nemoclaw-proxy-env.sh" /etc/profile.d/nemoclaw-proxy.sh 2>...
Step 74/79 : RUN sha256sum /sandbox/.openclaw/openclaw.json > /sandbox/.openclaw/.config-hash && chmod 660 ...
Step 74/79 completed in 0.4s (RUN sha256sum /sandbox/.openclaw/openclaw.json > /sandbox/.openclaw/.config-hash ...
Step 75/79 : RUN chown root:root /sandbox/.nemoclaw && chmod 1755 /sandbox/.nemoclaw && chown -R root:r...
Step 75/79 completed in 0.5s (RUN chown root:root /sandbox/.nemoclaw && chmod 1755 /sandbox/.nemoclaw && chown ...
Step 76/79 : RUN if [ "$NEMOCLAW_DARWIN_VM_COMPAT" = "1" ]; then chmod -R a+rwX /sandbox/.openclaw; ...
Step 76/79 completed in 0.4s (RUN if [ "$NEMOCLAW_DARWIN_VM_COMPAT" = "1" ]; then chmod -R a+rwX /sandbox/.open...
Step 77/79 : HEALTHCHECK --interval=30s --timeout=5s --start-period=45s --retries=3 CMD port="${NEMOCLAW_DA...
Step 77/79 completed in 0.1s (HEALTHCHECK --interval=30s --timeout=5s --start-period=45s --retries=3 CMD port="...
Step 78/79 : ENTRYPOINT ["/usr/local/bin/nemoclaw-start"]
Step 78/79 completed in 0.1s (ENTRYPOINT ["/usr/local/bin/nemoclaw-start"])
Step 79/79 : CMD ["/bin/bash"]
Step 79/79 completed in 0.1s (CMD ["/bin/bash"])
Sandbox image build completed in 44.7s
Creating sandbox in gateway...
Built image openshell/sandbox-from:1779882126
Waiting for sandbox to become ready...
Sandbox reported Ready before create stream exited; continuing.
Recreating OpenShell Docker sandbox container with NVIDIA GPU access...
✓ Docker GPU mode selected: --gpus all
Waiting for sandbox to become ready...
Waiting for NemoClaw dashboard to become ready...
Dashboard taking longer than expected to start. Continuing...
Verifying direct sandbox GPU access...
✗ GPU proof failed: nvidia-smi when available
Error: × sandbox 'my-assistant' is not ready (phase: Error); wait for it to reach │ Ready state
Install/configure NVIDIA Container Toolkit CDI, then restart Docker:
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
sudo systemctl restart docker
Or force CPU sandbox behavior with NEMOCLAW_SANDBOX_GPU=0.
Diagnostics saved: /home/masaki/.nemoclaw/onboard-failures/2026-05-27T11-43-34-431Z-my-assistant-docker-gpu-patch
The failed sandbox/container has been left in place for inspection.
Manual cleanup:
openshell sandbox delete "my-assistant"
/home/masaki/github/engchain_agent/NemoClaw/dist/lib/onboard/sandbox-gpu-preflight.js:140
throw new Error(`GPU proof failed: ${proof.label} (status ${statusText})${diagnosticSuffix}`);
^
Error: GPU proof failed: nvidia-smi when available (status 1): Error: × sandbox 'my-assistant' is not ready (phase: Error); wait for it to reach │ Ready state
at verifyDirectSandboxGpu (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/onboard/sandbox-gpu-preflight.js:140:19)
at Object.createSandbox (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/onboard.js:2921:13)
at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
at async handleSandboxState (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/onboard/machine/handlers/sandbox.js:169:23)
at async Object.onboard [as runOnboard] (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/onboard.js:5730:36)
at async runOnboardCommand (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/onboard/legacy-command.js:212:5)
at async runOnboardAction (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/actions/onboard.js:26:5)
at async runOnboardAction (/home/masaki/github/engchain_agent/NemoClaw/dist/lib/actions/global.js:28:5)
at async OnboardCliCommand.run (/home/masaki/github/engchain_agent/NemoClaw/dist/commands/onboard.js:18:9)
at async OnboardCliCommand._run (/home/masaki/github/engchain_agent/NemoClaw/node_modules/@oclif/core/lib/command.js:182:22)
Node.js v22.22.3
Environment
- CPU: Intel Core-i9 12900KS
- memory: 96 GB
- storage: SATA SSD 500 GB (160 GB free)
- GPU: NVIDIA Geforce RTX 4090 * 1
- OS: Ubuntu 22.04 LTS
- NVIDIA driver: v565.77
- Python: v3.10.12
- Ollama: v0.20.4
- LLM: GPT-OSS:20B
- docker engine: v27.3.1
usable without sudo
- NVIDIA Container Toolkit: v1.18.1
- Node.js: v22.22.3
- npm: v10.9.8
Debug Output
Logs
Checklist
Description
I can use NVIDIA GPUs from normal Docker containers with
--gpus all, but nomeclaw onbording with--gpufails after the sandbox container is created with GPU access.Reproduction Steps
Docker and nvidia-ctk are configurated correctly.
However, nemoclaw omboaring is failed as below.
Environment
Debug Output
Logs
Checklist