Description

================================================================================
PROBLEM: OpenClaw GatewayRequestError — EACCES permission denied

ERROR:
GatewayRequestError: Error: EACCES: permission denied,
open '/sandbox/.openclaw/exec-approvals.json': code=EACCES

ENVIRONMENT:

• Sandbox: nemo-tech01 (via nemoclaw / openshell)
• Cluster container: openshell-cluster-nemoclaw (ghcr.io/nvidia/openshell/cluster:0.0.16)
• Container ID on host: 9c3897f8d232
• Sandbox user: sandbox (uid=998, gid=998)
• OpenClaw version: 2026.3.11
• Sandbox rootfs inside cluster:
  /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/
  548aebc85e7313b3daaf81cb460bde1f0ee779fc236af2283589b66ac8076beb/rootfs

ROOT CAUSE (multiple issues):

1. /sandbox/.openclaw/ directory owned by root:root (permissions 0755)
   — sandbox user (uid 998) has read+execute but NOT write access
2. exec-approvals.json file did not exist at all, and its symlink was missing
3. openclaw.json owned by root with 0444 — read-only, gateway can't update config
4. openclaw.json.bak owned by root with 0600 — sandbox can't even READ it
5. .config-hash owned by root with 0444 — sandbox can't update the hash
6. OpenClaw gateway (running as sandbox user) needs read+write to all of these

OBSERVATION:
The /sandbox/.openclaw/ dir uses a two-layer design:
- /sandbox/.openclaw/ → root-managed, contains config files + symlinks
- /sandbox/.openclaw-data/ → sandbox-owned, writable, target of symlinks
Most entries (agents, canvas, cron, devices, extensions, hooks, identity,
skills, update-check.json, workspace) are symlinks from .openclaw → .openclaw-data.
But exec-approvals.json symlink was MISSING, and the static files (openclaw.json,
openclaw.json.bak, .config-hash) had overly restrictive root ownership.

================================================================================
STEPS THAT WORKED (in order)

STEP 1 — Set up SSH access to the sandbox (from host)
$ openshell sandbox ssh-config nemo-tech01 >> ~/.ssh/config
$ ssh openshell-nemo-tech01 "whoami"

Output: sandbox

STEP 2 — Diagnose: inspect permissions from inside sandbox
$ ssh openshell-nemo-tech01 "ls -la /sandbox/.openclaw/"

Showed: dir=root:root, openclaw.json=root:root(444),

openclaw.json.bak=root:root(600), no exec-approvals.json

STEP 3 — Identify the sandbox rootfs from the cluster container
$ docker exec openshell-cluster-nemoclaw sh -c '
for rootfs in /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/*/rootfs; do
if [ -f "$rootfs/sandbox/.openclaw/openclaw.json" ]; then
agents=$(ls "$rootfs/sandbox/.openclaw-data/agents/" 2>/dev/null)
has_data=$(ls "$rootfs/sandbox/.openclaw-data/exec-approvals.json" 2>/dev/null && echo "HAS" || echo "MISSING")
echo "rootfs=$rootfs agents=$agents exec_approvals=$has_data"
fi
done'

Result: identified rootfs=...548aebc85e73.../rootfs as nemo-tech01

(matched by checking which one had our uploaded test file in .openclaw-data)

STEP 4 — Fix directory ownership + create exec-approvals.json symlink (from cluster as root)
$ docker exec openshell-cluster-nemoclaw sh -c '
ROOTFS="/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/548aebc85e7313b3daaf81cb460bde1f0ee779fc236af2283589b66ac8076beb/rootfs"
chown 998:998 "$ROOTFS/sandbox/.openclaw/"
chmod u+rwx "$ROOTFS/sandbox/.openclaw/"
ln -sf /sandbox/.openclaw-data/exec-approvals.json "$ROOTFS/sandbox/.openclaw/exec-approvals.json"
chown -R 998:998 "$ROOTFS/sandbox/.openclaw/logs" 2>/dev/null'

STEP 5 — Create the actual exec-approvals.json file in the writable data directory
$ ssh openshell-nemo-tech01
"rm -rf /sandbox/.openclaw-data/exec-approvals.json &&
echo '{}' > /sandbox/.openclaw-data/exec-approvals.json"

NOTE: The initial openshell sandbox upload command created exec-approvals.json
as a DIRECTORY (tar extraction artifact), so we had to rm -rf and recreate
it as a proper file.

STEP 6 — Fix remaining root-owned files (from cluster as root)
$ docker exec openshell-cluster-nemoclaw sh -c '
ROOTFS="/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/548aebc85e7313b3daaf81cb460bde1f0ee779fc236af2283589b66ac8076beb/rootfs"
chown 998:998 "$ROOTFS/sandbox/.openclaw/openclaw.json"
chown 998:998 "$ROOTFS/sandbox/.openclaw/openclaw.json.bak"
chown 998:998 "$ROOTFS/sandbox/.openclaw/.config-hash"
chmod u+rw "$ROOTFS/sandbox/.openclaw/openclaw.json"
chmod u+rw "$ROOTFS/sandbox/.openclaw/openclaw.json.bak"
chmod u+rw "$ROOTFS/sandbox/.openclaw/.config-hash"'

This fixed:
openclaw.json : root:root r--r--r-- → sandbox:sandbox rw-r--r--
openclaw.json.bak : root:root rw------- → sandbox:sandbox rw-------
.config-hash : root:root r--r--r-- → sandbox:sandbox rw-r--r--

STEP 7 — Verify EVERYTHING from inside sandbox
$ ssh openshell-nemo-tech01 "
cat /sandbox/.openclaw/openclaw.json > /dev/null && echo 'openclaw.json: READ OK';
cat /sandbox/.openclaw/openclaw.json.bak > /dev/null && echo 'openclaw.json.bak: READ OK';
cat /sandbox/.openclaw/.config-hash > /dev/null && echo '.config-hash: READ OK';
cat /sandbox/.openclaw/exec-approvals.json > /dev/null && echo 'exec-approvals.json: READ OK';
touch /sandbox/.openclaw/test-file && rm /sandbox/.openclaw/test-file && echo 'Dir write: OK';
echo '{}' > /sandbox/.openclaw/test-write && rm /sandbox/.openclaw/test-write && echo 'File create: OK'"

All passed ✓

================================================================================
FINAL STATE (verified)

/sandbox/.openclaw/ ownership: sandbox:sandbox perms: drwxr-xr-x ✓
.config-hash sandbox:sandbox rw-r--r-- ✓ (was root 444)
agents/ symlink → .openclaw-data/agents ✓
canvas/ symlink → .openclaw-data/canvas ✓
cron/ symlink → .openclaw-data/cron ✓
devices/ symlink → .openclaw-data/devices ✓
exec-approvals.json symlink → .openclaw-data/exec-approvals.json ✓ (was MISSING)
extensions/ symlink → .openclaw-data/extensions ✓
hooks/ symlink → .openclaw-data/hooks ✓
identity/ symlink → .openclaw-data/identity ✓
logs/ sandbox:sandbox drwx------ ✓
openclaw.json sandbox:sandbox rw-r--r-- ✓ (was root 444)
openclaw.json.bak sandbox:sandbox rw------- ✓ (was root 600)
skills/ symlink → .openclaw-data/skills ✓
update-check.json symlink → .openclaw-data/update-check.json ✓
workspace/ symlink → .openclaw-data/workspace ✓

================================================================================
SUMMARY

The fix required going through the cluster container (docker exec on
openshell-cluster-nemoclaw) because:

• The sandbox has NO sudo command
• The /sandbox/.openclaw/ dir and its files were root-owned
• Only the cluster container has root access to the sandbox's overlay FS
  (k3s containerd at /run/k3s/containerd/.../rootfs)

Things that were fixed:

1. Directory ownership: /sandbox/.openclaw/ → chown to sandbox (998)
2. Missing file: exec-approvals.json → created via symlink to .openclaw-data/
3. Config files: openclaw.json, openclaw.json.bak, .config-hash → chown to sandbox
4. Logs dir: /sandbox/.openclaw/logs/ → chown to sandbox

Key commands path:
docker exec (on openshell-cluster-nemoclaw, as root) → chown + chmod + symlink
ssh (into sandbox via openshell-nemo-tech01, as sandbox user) → create JSON file

GOTCHA — openshell sandbox upload creates a DIRECTORY not a file:
openshell sandbox upload nemo-tech01 file.json /path/file.json
creates /path/file.json/ as a directory (tar artifact). Must rm -rf and
recreate manually as a file via ssh.

NOTE — This fix may need to be re-applied if the sandbox is recreated/restarted,
since the overlay filesystem changes may not persist across container rebuilds.

================================================================================
ENVIRONMENT DETAILS

Host Machine:

• OS: Ubuntu 22.04.4 LTS (Jammy Jellyfish)
• Kernel: Linux 5.15.0-113-generic x86_64
• Hostname: gccvmgsecc02

Versions:

• NemoClaw: v0.1.0 (output of nemoclaw --version)
• OpenShell: 0.0.16 (output of openshell --version)
• Node.js: v20.20.2 (output of node --version)
• Docker: 27.0.1 (Docker Engine, output of docker --version)
• OpenClaw: 2026.3.11 (shown in openclaw TUI banner inside sandbox)

Cluster Container:

• Image: ghcr.io/nvidia/openshell/cluster:0.0.16
• Container name: openshell-cluster-nemoclaw
• Container ID: 9c3897f8d232
• Internal runtime: k3s with containerd (overlay filesystem)

Sandboxes:

• nemo-tech (default) — model: unknown, provider: unknown, GPU, policies: pypi, npm
• nemo-tech01 — model: unknown, provider: unknown, GPU, policies: pypi, npm
• Sandbox user: sandbox (uid=998, gid=998)
• No sudo available inside sandbox
• Inference model: qwen3:8b (via Ollama)

Port Forwarding:

• Sandbox access: port 18789 forwarded to nemo-tech01
  $ openshell forward start -d 18789 nemo-tech01
• OpenClaw dashboard accessible via forwarded port

================================================================================
STEPS TO REPRODUCE THE BUG

Prerequisites:

1. NemoClaw installed and running (nemoclaw list shows sandboxes)

2. OpenShell cluster container running:
   $ docker ps | grep openshell

   should show openshell-cluster-nemoclaw

3. At least one sandbox created:
   $ nemoclaw list

   should show nemo-tech01 (or any sandbox name)

Reproduce:

1. Connect to the sandbox:
   $ nemoclaw nemo-tech01 connect

   or

   $ openshell forward start -d 18789 nemo-tech01

2. Open the OpenClaw dashboard in a browser:
   http://127.0.0.1:18789/

   Navigate to Nodes page

3. Observe the error banner:
   GatewayRequestError: Error: EACCES: permission denied,
   open '/sandbox/.openclaw/exec-approvals.json': code=EACCES

4. Confirm from inside the sandbox:
   $ ssh openshell-nemo-tech01 "ls -la /sandbox/.openclaw/"

   Directory owned by root:root — sandbox user cannot create files

   exec-approvals.json does not exist and cannot be created

5. Also verify no sudo:
   $ ssh openshell-nemo-tech01 "sudo ls"

   Output: bash: sudo: command not found

Why it happens:

• The openshell cluster image (0.0.16) creates /sandbox/.openclaw/ as root
• It sets up symlinks for most things pointing to the writable
  /sandbox/.openclaw-data/ directory
• But the exec-approvals.json symlink is NOT created during sandbox provisioning
• The directory itself remains root-owned (0755), blocking file creation
• The gateway process runs as user sandbox (998) and fails on first access

How to verify the fix worked:

1. Refresh the OpenClaw dashboard — error banner should be gone
2. From sandbox SSH:
   $ ssh openshell-nemo-tech01 "cat /sandbox/.openclaw/exec-approvals.json"

   Should show JSON content (not "Permission denied")

3. From sandbox SSH — verify write:
   $ ssh openshell-nemo-tech01 "echo test > /sandbox/.openclaw/test && rm /sandbox/.openclaw/test && echo OK"

   Should output: OK

Reproduction Steps

GatewayRequestError: Error: EACCES: permission denied, open '/sandbox/.openclaw/exec-approvals.json': code=EACCES

Environment

ENVIRONMENT:

• Sandbox: nemo-tech01 (via nemoclaw / openshell)
• Cluster container: openshell-cluster-nemoclaw (ghcr.io/nvidia/openshell/cluster:0.0.16)
• Container ID on host: 9c3897f8d232
• Sandbox user: sandbox (uid=998, gid=998)
• OpenClaw version: 2026.3.11
• Sandbox rootfs inside cluster:
  /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/
  548aebc85e7313b3daaf81cb460bde1f0ee779fc236af2283589b66ac8076beb/rootfs

Debug Output

GatewayRequestError: Error: EACCES: permission denied, open '/sandbox/.openclaw/exec-approvals.json': code=EACCES

Logs

GatewayRequestError: Error: EACCES: permission denied, open '/sandbox/.openclaw/exec-approvals.json': code=EACCES

Checklist

• I confirmed this bug is reproducible
• I searched existing issues and this is not a duplicate