Skip to content

test: add regression guards for sandbox config provisioning (#1027, #514)#1236

Merged
ericksoa merged 3 commits into
NVIDIA:mainfrom
TonyLuo-NV:fix/1027-exec-approvals-symlink
Apr 24, 2026
Merged

test: add regression guards for sandbox config provisioning (#1027, #514)#1236
ericksoa merged 3 commits into
NVIDIA:mainfrom
TonyLuo-NV:fix/1027-exec-approvals-symlink

Conversation

@TonyLuo-NV

@TonyLuo-NV TonyLuo-NV commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Reboot of this PR after #1519 merged the actual code fix for issue #1027. This branch is now test-only — it adds static regression guards so a future refactor that drops the provisioning steps fails fast in CI, before a docker build runs.

The original #1236 carried the code fix alongside tests; since upstream already has the fix from #1519, I rebased to upstream/main and kept only the missing regression coverage.

What this adds

test/sandbox-provisioning.test.ts — 8 static assertions on the image-build sources:

#1027 / #1519 (runtime-writable symlinks)

  • Dockerfile.base creates the exec-approvals.json backing file in .openclaw-data
  • Dockerfile.base symlinks .openclaw/exec-approvals.json.openclaw-data/exec-approvals.json
  • Same pair for update-check.json
  • Ordering guard: the data file is created before the symlink that points at it

#514 (root-owned read-only config)

  • Dockerfile keeps openclaw.json at mode 0444 (agent cannot tamper with auth token / CORS)
  • Dockerfile keeps .config-hash at root:root 0444 (agent cannot forge integrity hash)
  • Dockerfile keeps .openclaw/ at root:root 0755 (agent cannot replace symlinks)

Why these tests matter

  • The existing test/exec-approvals-path-regression.test.ts only verifies the path-patching grep logic in Dockerfile.base, not the actual symlink/data file creation
  • The existing test/e2e-gateway-isolation.sh catches regressions at image-build time, but requires docker and a full build; static tests fail in seconds on every PR
  • fix(image): add missing openclaw-data symlinks for runtime-writable paths #1519 was a 6-line code fix with zero test coverage. If a future cleanup of Dockerfile.base drops those six lines, the issue returns silently

Test plan

  • npx vitest run test/sandbox-provisioning.test.ts — 8/8 passing
  • Rebased onto current upstream/main (d9aced49)
  • No code changes; test-only PR

Signed-off-by: Tony Luo xialuo@nvidia.com

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Apr 1, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds a writable sandbox state file and symlink under .openclaw-data, updates Dockerfile comments to reflect runtime-modifiable config and DAC guidance, and expands tests to assert ownership/permissions and symlink targets for .openclaw and .openclaw-data entries.

Changes

Cohort / File(s) Summary
Dockerfile (comments)
Dockerfile
Updated comment block: acknowledges /sandbox/.openclaw/openclaw.json may be modified at runtime by the gateway, broadens DAC hardening guidance to protect the .openclaw directory/config files while runtime state lives in .openclaw-data, and updates issue reference from #514 to #1027. No executable changes.
Sandbox state provisioning
Dockerfile.base
Adds writable state file /sandbox/.openclaw-data/exec-approvals.json initialized to {} and creates symlink ln -s /sandbox/.openclaw-data/exec-approvals.json /sandbox/.openclaw/exec-approvals.json; included in existing chown -R sandbox:sandbox /sandbox/.openclaw /sandbox/.openclaw-data.
Shell e2e tests
test/e2e-gateway-isolation.sh
Replaces prior "sandbox cannot write" touch/echo checks with explicit stat assertions that /sandbox/.openclaw/openclaw.json and /sandbox/.openclaw/.config-hash are root:root mode 444; extends symlink target checks to include memory, update-check.json, and exec-approvals.json; adds tests to read and write exec-approvals.json via symlink, conditional check for openclaw.json.bak ownership, and asserts /sandbox/.openclaw is root:root mode 755.
Unit/regression tests
test/sandbox-provisioning.test.js
New Vitest file validating Dockerfile.base creates backing data file for exec-approvals.json and then the symlink, asserts symlink presence for update-check.json and others, and checks Dockerfile contains commands to set /sandbox/.openclaw/openclaw.json and /sandbox/.openclaw/.config-hash to chmod 444 and chown root:root, and /sandbox/.openclaw to chown root:root + chmod 755.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I nested files where roots can’t pry,
A tiny symlink where permissions lie,
I guard the hashes, tuck approvals near,
Tests hop the paths and check what's clear,
The sandbox hums, secure and sly.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main changes: adding regression tests for sandbox config provisioning and addressing issues #1027 and #514.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/sandbox-provisioning.test.js (1)

68-83: Consider extracting fs.readFileSync to reduce redundant I/O.

The Dockerfile content is read separately in each parameterized test iteration. While this works correctly, reading once at describe-block scope would be cleaner and slightly faster.

♻️ Optional: Extract file read to describe scope
 describe("sandbox provisioning: all expected symlinks present in Dockerfile.base", () => {
   const EXPECTED_DIR_SYMLINKS = [
     "agents", "extensions", "workspace", "skills", "hooks",
     "identity", "devices", "canvas", "cron", "memory",
   ];
   const EXPECTED_FILE_SYMLINKS = ["update-check.json", "exec-approvals.json"];
+  const dockerfileBaseSrc = fs.readFileSync(DOCKERFILE_BASE, "utf-8");

   it.each(EXPECTED_DIR_SYMLINKS)("symlink for directory '%s' is created", (name) => {
-    const src = fs.readFileSync(DOCKERFILE_BASE, "utf-8");
-    expect(src).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`);
+    expect(dockerfileBaseSrc).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`);
   });

   it.each(EXPECTED_FILE_SYMLINKS)("symlink for file '%s' is created", (name) => {
-    const src = fs.readFileSync(DOCKERFILE_BASE, "utf-8");
-    expect(src).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`);
+    expect(dockerfileBaseSrc).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`);
   });
 });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/sandbox-provisioning.test.js` around lines 68 - 83, Move the
fs.readFileSync(DOCKERFILE_BASE, "utf-8") call out of each it.each and into the
surrounding describe scope so the Dockerfile contents are read once; create a
local const (e.g. dockerfileSrc) in the describe block and replace the per-test
reads in the it.each blocks that reference EXPECTED_DIR_SYMLINKS and
EXPECTED_FILE_SYMLINKS to use dockerfileSrc instead.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/sandbox-provisioning.test.js`:
- Around line 68-83: Move the fs.readFileSync(DOCKERFILE_BASE, "utf-8") call out
of each it.each and into the surrounding describe scope so the Dockerfile
contents are read once; create a local const (e.g. dockerfileSrc) in the
describe block and replace the per-test reads in the it.each blocks that
reference EXPECTED_DIR_SYMLINKS and EXPECTED_FILE_SYMLINKS to use dockerfileSrc
instead.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d7b6ef09-ce6d-4b06-a7e3-0ab5405c0f28

📥 Commits

Reviewing files that changed from the base of the PR and between e06c147 and e4adb21.

📒 Files selected for processing (4)
  • Dockerfile
  • Dockerfile.base
  • test/e2e-gateway-isolation.sh
  • test/sandbox-provisioning.test.js

@TonyLuo-NV TonyLuo-NV force-pushed the fix/1027-exec-approvals-symlink branch from e4adb21 to 8edce8e Compare April 1, 2026 07:39
@TonyLuo-NV

Copy link
Copy Markdown
Contributor Author

CI failure analysis: test-e2e-gateway-isolation

The exec-approvals.json symlink test (test 9) fails because the CI uses the cached GHCR base image (ghcr.io/nvidia/nemoclaw/sandbox-base:latest) which does not contain the Dockerfile.base changes from this PR.

Root cause: .github/actions/resolve-sandbox-base-image/action.yaml always pulls the GHCR cached base image when available, without checking whether Dockerfile.base was modified in the PR. The base image is only rebuilt on main merges via base-image.yaml.

Suggested fix for the CI action: detect Dockerfile.base changes in the PR (e.g., git diff --name-only origin/main...HEAD -- Dockerfile.base) and force a local build when it has been modified. This would be an isolated change to .github/actions/resolve-sandbox-base-image/action.yaml.

The dco-check failure is expected — sign-off will be added after QA review.

@wscurran wscurran added the bug Something fails against expected or documented behavior label Apr 1, 2026
@TonyLuo-NV TonyLuo-NV assigned TonyLuo-NV and unassigned TonyLuo-NV Apr 2, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
test/e2e-gateway-isolation.sh (1)

193-206: Consider unifying the assertion style with Tests 2 and 5.

Test 9d splits the stat output with awk for separate owner/permissions checks, while Tests 2 and 5 use exact string matching. Both approaches are correct, but unifying them would improve maintainability.

♻️ Optional: unified approach matching Tests 2/5
 info "9d. /sandbox/.openclaw/ directory is owned by root:root with permissions 755"
 OUT=$(run_as_root "stat -c '%U:%G %a' /sandbox/.openclaw")
-OWNER=$(echo "$OUT" | awk '{print $1}')
-DIR_PERMS=$(echo "$OUT" | awk '{print $2}')
-if [ "$OWNER" = "root:root" ]; then
-  pass ".openclaw directory owned by root:root"
+if [ "$OUT" = "root:root 755" ]; then
+  pass ".openclaw directory is root:root 755"
 else
-  fail ".openclaw directory owner wrong: $OWNER (expected root:root)"
-fi
-if [ "$DIR_PERMS" = "755" ]; then
-  pass ".openclaw directory permissions are 755"
-else
-  fail ".openclaw directory permissions wrong: $DIR_PERMS (expected 755)"
+  fail ".openclaw directory wrong: $OUT (expected root:root 755)"
 fi
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e-gateway-isolation.sh` around lines 193 - 206, Unify Test 9d to use
an exact string match like Tests 2 and 5 by comparing the full stat output
stored in OUT (from run_as_root "stat -c '%U:%G %a' /sandbox/.openclaw") against
an expected string "root:root 755"; replace the separate OWNER/DIR_PERMS awk
parsing and two checks with a single if [ "$OUT" = "root:root 755" ] then pass
".openclaw owned and perms OK" else fail ".openclaw owner/perms wrong: $OUT
(expected root:root 755)" fi, updating the pass/fail messages and keeping
references to OUT, run_as_root, pass, and fail.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@Dockerfile`:
- Around line 80-82: Update the misleading Dockerfile comment: clarify that
openclaw.json is immutable at runtime (root:root 444) and cannot be modified by
the gateway process, and state that runtime exec approvals are stored in
exec-approvals.json (symlinked) rather than openclaw.json; keep the note about
Landlock/root-owned directory as the primary security boundary but remove the
claim that the gateway updates openclaw.json for exec approvals.

---

Nitpick comments:
In `@test/e2e-gateway-isolation.sh`:
- Around line 193-206: Unify Test 9d to use an exact string match like Tests 2
and 5 by comparing the full stat output stored in OUT (from run_as_root "stat -c
'%U:%G %a' /sandbox/.openclaw") against an expected string "root:root 755";
replace the separate OWNER/DIR_PERMS awk parsing and two checks with a single if
[ "$OUT" = "root:root 755" ] then pass ".openclaw owned and perms OK" else fail
".openclaw owner/perms wrong: $OUT (expected root:root 755)" fi, updating the
pass/fail messages and keeping references to OUT, run_as_root, pass, and fail.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6bc6fb3d-9e3b-49e1-8856-70280dc9c8fc

📥 Commits

Reviewing files that changed from the base of the PR and between e4adb21 and 648be6c.

📒 Files selected for processing (4)
  • Dockerfile
  • Dockerfile.base
  • test/e2e-gateway-isolation.sh
  • test/sandbox-provisioning.test.js
✅ Files skipped from review due to trivial changes (1)
  • Dockerfile.base
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/sandbox-provisioning.test.js

Comment thread Dockerfile Outdated
@TonyLuo-NV TonyLuo-NV force-pushed the fix/1027-exec-approvals-symlink branch from 648be6c to a1d8a94 Compare April 2, 2026 09:19
TonyLuo-NV added a commit to TonyLuo-NV/NemoClaw that referenced this pull request Apr 2, 2026
- Correct misleading Dockerfile comment: openclaw.json is immutable at
  runtime (root:root 444), not updated by the gateway. Exec approvals
  live in exec-approvals.json via symlink.
- Extract fs.readFileSync to describe-block scope in sandbox-provisioning
  tests to eliminate redundant I/O per test iteration.
- Unify assertion style in e2e-gateway-isolation.sh Test 9d to use exact
  string matching consistent with Tests 2 and 5.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wscurran

Copy link
Copy Markdown
Contributor

Thanks for the exec-approvals symlink fix. The codebase has changed significantly since April 1 — including a full TypeScript migration — so this will need a rebase on origin/main before we can review it. Please rebase and resolve any conflicts, and we'll take a look.

, NVIDIA#514)

The original code fix for GatewayRequestError EACCES on exec-approvals.json
landed via NVIDIA#1519, which added just the two symlink/touch lines in
Dockerfile.base without any regression tests. This adds static assertions
on the image-build sources so a future refactor that drops the
provisioning steps fails fast in CI, before a docker build runs.

Covers:
- NVIDIA#1027/NVIDIA#1519: exec-approvals.json + update-check.json backing files and
  symlinks in Dockerfile.base (including ordering: data file created
  before the symlink that points at it).
- NVIDIA#514: openclaw.json stays mode 0444, .config-hash stays root:root 0444,
  and /sandbox/.openclaw/ itself stays root:root 0755 so the agent cannot
  replace symlinks or forge an integrity hash.

Signed-off-by: Tony Luo <xialuo@nvidia.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@TonyLuo-NV TonyLuo-NV force-pushed the fix/1027-exec-approvals-symlink branch from 6d015fb to 5ad3442 Compare April 23, 2026 09:37
@copy-pr-bot

copy-pr-bot Bot commented Apr 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@TonyLuo-NV TonyLuo-NV changed the title fix(sandbox): add missing exec-approvals.json symlink test: add regression guards for sandbox config provisioning (#1027, #514) Apr 23, 2026
@TonyLuo-NV TonyLuo-NV linked an issue Apr 23, 2026 that may be closed by this pull request
2 tasks

@ericksoa ericksoa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean test-only PR. Static Dockerfile assertions that fail in seconds vs waiting for a full image build are a good complement to the e2e tests.

The 8 guards cover the right invariants: symlink layout (#1027/#1519), ordering (data file before symlink), and root-owned read-only config (#514).

One housekeeping note: the status: rebase label looks stale now that this has been rebased onto current main. May want to remove it before merge.

LGTM.

@ericksoa ericksoa enabled auto-merge (squash) April 24, 2026 00:05
@ericksoa ericksoa merged commit fbe5858 into NVIDIA:main Apr 24, 2026
12 checks passed
DemianHeyGen pushed a commit to DemianHeyGen/NemoClaw that referenced this pull request Apr 30, 2026
, NVIDIA#514) (NVIDIA#1236)

## Summary

Reboot of this PR after NVIDIA#1519 merged the actual code fix for issue
NVIDIA#1027. This branch is now **test-only** — it adds static regression
guards so a future refactor that drops the provisioning steps fails fast
in CI, before a docker build runs.

The original NVIDIA#1236 carried the code fix alongside tests; since upstream
already has the fix from NVIDIA#1519, I rebased to upstream/main and kept only
the missing regression coverage.

## What this adds

`test/sandbox-provisioning.test.ts` — 8 static assertions on the
image-build sources:

**NVIDIA#1027 / NVIDIA#1519 (runtime-writable symlinks)**
- `Dockerfile.base` creates the `exec-approvals.json` backing file in
`.openclaw-data`
- `Dockerfile.base` symlinks `.openclaw/exec-approvals.json` →
`.openclaw-data/exec-approvals.json`
- Same pair for `update-check.json`
- Ordering guard: the data file is created before the symlink that
points at it

**NVIDIA#514 (root-owned read-only config)**
- `Dockerfile` keeps `openclaw.json` at mode 0444 (agent cannot tamper
with auth token / CORS)
- `Dockerfile` keeps `.config-hash` at root:root 0444 (agent cannot
forge integrity hash)
- `Dockerfile` keeps `.openclaw/` at root:root 0755 (agent cannot
replace symlinks)

## Why these tests matter

- The existing `test/exec-approvals-path-regression.test.ts` only
verifies the path-patching grep logic in `Dockerfile.base`, not the
actual symlink/data file creation
- The existing `test/e2e-gateway-isolation.sh` catches regressions at
image-build time, but requires docker and a full build; static tests
fail in seconds on every PR
- NVIDIA#1519 was a 6-line code fix with zero test coverage. If a future
cleanup of `Dockerfile.base` drops those six lines, the issue returns
silently

## Test plan

- [x] `npx vitest run test/sandbox-provisioning.test.ts` — 8/8 passing
- [x] Rebased onto current `upstream/main` (`d9aced49`)
- [x] No code changes; test-only PR

Signed-off-by: Tony Luo <xialuo@nvidia.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Tony Luo <xialuo@nvidia.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Aaron Erickson 🦞 <aerickson@nvidia.com>
@TonyLuo-NV TonyLuo-NV deleted the fix/1027-exec-approvals-symlink branch May 6, 2026 08:14
@wscurran wscurran added bug-fix PR fixes a bug or regression needs: rebase PR needs rebase or conflict resolution and removed status: rebase bug Something fails against expected or documented behavior labels Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression needs: rebase PR needs rebase or conflict resolution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenClaw GatewayRequestError — EACCES permission denied

3 participants