test: add regression guards for sandbox config provisioning (#1027, #514)#1236
Conversation
📝 WalkthroughWalkthroughAdds a writable sandbox state file and symlink under Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
test/sandbox-provisioning.test.js (1)
68-83: Consider extractingfs.readFileSyncto reduce redundant I/O.The Dockerfile content is read separately in each parameterized test iteration. While this works correctly, reading once at describe-block scope would be cleaner and slightly faster.
♻️ Optional: Extract file read to describe scope
describe("sandbox provisioning: all expected symlinks present in Dockerfile.base", () => { const EXPECTED_DIR_SYMLINKS = [ "agents", "extensions", "workspace", "skills", "hooks", "identity", "devices", "canvas", "cron", "memory", ]; const EXPECTED_FILE_SYMLINKS = ["update-check.json", "exec-approvals.json"]; + const dockerfileBaseSrc = fs.readFileSync(DOCKERFILE_BASE, "utf-8"); it.each(EXPECTED_DIR_SYMLINKS)("symlink for directory '%s' is created", (name) => { - const src = fs.readFileSync(DOCKERFILE_BASE, "utf-8"); - expect(src).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`); + expect(dockerfileBaseSrc).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`); }); it.each(EXPECTED_FILE_SYMLINKS)("symlink for file '%s' is created", (name) => { - const src = fs.readFileSync(DOCKERFILE_BASE, "utf-8"); - expect(src).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`); + expect(dockerfileBaseSrc).toContain(`ln -s /sandbox/.openclaw-data/${name} /sandbox/.openclaw/${name}`); }); });🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/sandbox-provisioning.test.js` around lines 68 - 83, Move the fs.readFileSync(DOCKERFILE_BASE, "utf-8") call out of each it.each and into the surrounding describe scope so the Dockerfile contents are read once; create a local const (e.g. dockerfileSrc) in the describe block and replace the per-test reads in the it.each blocks that reference EXPECTED_DIR_SYMLINKS and EXPECTED_FILE_SYMLINKS to use dockerfileSrc instead.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@test/sandbox-provisioning.test.js`:
- Around line 68-83: Move the fs.readFileSync(DOCKERFILE_BASE, "utf-8") call out
of each it.each and into the surrounding describe scope so the Dockerfile
contents are read once; create a local const (e.g. dockerfileSrc) in the
describe block and replace the per-test reads in the it.each blocks that
reference EXPECTED_DIR_SYMLINKS and EXPECTED_FILE_SYMLINKS to use dockerfileSrc
instead.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: d7b6ef09-ce6d-4b06-a7e3-0ab5405c0f28
📒 Files selected for processing (4)
DockerfileDockerfile.basetest/e2e-gateway-isolation.shtest/sandbox-provisioning.test.js
e4adb21 to
8edce8e
Compare
CI failure analysis:
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
test/e2e-gateway-isolation.sh (1)
193-206: Consider unifying the assertion style with Tests 2 and 5.Test 9d splits the stat output with awk for separate owner/permissions checks, while Tests 2 and 5 use exact string matching. Both approaches are correct, but unifying them would improve maintainability.
♻️ Optional: unified approach matching Tests 2/5
info "9d. /sandbox/.openclaw/ directory is owned by root:root with permissions 755" OUT=$(run_as_root "stat -c '%U:%G %a' /sandbox/.openclaw") -OWNER=$(echo "$OUT" | awk '{print $1}') -DIR_PERMS=$(echo "$OUT" | awk '{print $2}') -if [ "$OWNER" = "root:root" ]; then - pass ".openclaw directory owned by root:root" +if [ "$OUT" = "root:root 755" ]; then + pass ".openclaw directory is root:root 755" else - fail ".openclaw directory owner wrong: $OWNER (expected root:root)" -fi -if [ "$DIR_PERMS" = "755" ]; then - pass ".openclaw directory permissions are 755" -else - fail ".openclaw directory permissions wrong: $DIR_PERMS (expected 755)" + fail ".openclaw directory wrong: $OUT (expected root:root 755)" fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e-gateway-isolation.sh` around lines 193 - 206, Unify Test 9d to use an exact string match like Tests 2 and 5 by comparing the full stat output stored in OUT (from run_as_root "stat -c '%U:%G %a' /sandbox/.openclaw") against an expected string "root:root 755"; replace the separate OWNER/DIR_PERMS awk parsing and two checks with a single if [ "$OUT" = "root:root 755" ] then pass ".openclaw owned and perms OK" else fail ".openclaw owner/perms wrong: $OUT (expected root:root 755)" fi, updating the pass/fail messages and keeping references to OUT, run_as_root, pass, and fail.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@Dockerfile`:
- Around line 80-82: Update the misleading Dockerfile comment: clarify that
openclaw.json is immutable at runtime (root:root 444) and cannot be modified by
the gateway process, and state that runtime exec approvals are stored in
exec-approvals.json (symlinked) rather than openclaw.json; keep the note about
Landlock/root-owned directory as the primary security boundary but remove the
claim that the gateway updates openclaw.json for exec approvals.
---
Nitpick comments:
In `@test/e2e-gateway-isolation.sh`:
- Around line 193-206: Unify Test 9d to use an exact string match like Tests 2
and 5 by comparing the full stat output stored in OUT (from run_as_root "stat -c
'%U:%G %a' /sandbox/.openclaw") against an expected string "root:root 755";
replace the separate OWNER/DIR_PERMS awk parsing and two checks with a single if
[ "$OUT" = "root:root 755" ] then pass ".openclaw owned and perms OK" else fail
".openclaw owner/perms wrong: $OUT (expected root:root 755)" fi, updating the
pass/fail messages and keeping references to OUT, run_as_root, pass, and fail.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 6bc6fb3d-9e3b-49e1-8856-70280dc9c8fc
📒 Files selected for processing (4)
DockerfileDockerfile.basetest/e2e-gateway-isolation.shtest/sandbox-provisioning.test.js
✅ Files skipped from review due to trivial changes (1)
- Dockerfile.base
🚧 Files skipped from review as they are similar to previous changes (1)
- test/sandbox-provisioning.test.js
648be6c to
a1d8a94
Compare
- Correct misleading Dockerfile comment: openclaw.json is immutable at runtime (root:root 444), not updated by the gateway. Exec approvals live in exec-approvals.json via symlink. - Extract fs.readFileSync to describe-block scope in sandbox-provisioning tests to eliminate redundant I/O per test iteration. - Unify assertion style in e2e-gateway-isolation.sh Test 9d to use exact string matching consistent with Tests 2 and 5. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Thanks for the exec-approvals symlink fix. The codebase has changed significantly since April 1 — including a full TypeScript migration — so this will need a rebase on |
, NVIDIA#514) The original code fix for GatewayRequestError EACCES on exec-approvals.json landed via NVIDIA#1519, which added just the two symlink/touch lines in Dockerfile.base without any regression tests. This adds static assertions on the image-build sources so a future refactor that drops the provisioning steps fails fast in CI, before a docker build runs. Covers: - NVIDIA#1027/NVIDIA#1519: exec-approvals.json + update-check.json backing files and symlinks in Dockerfile.base (including ordering: data file created before the symlink that points at it). - NVIDIA#514: openclaw.json stays mode 0444, .config-hash stays root:root 0444, and /sandbox/.openclaw/ itself stays root:root 0755 so the agent cannot replace symlinks or forge an integrity hash. Signed-off-by: Tony Luo <xialuo@nvidia.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6d015fb to
5ad3442
Compare
ericksoa
left a comment
There was a problem hiding this comment.
Clean test-only PR. Static Dockerfile assertions that fail in seconds vs waiting for a full image build are a good complement to the e2e tests.
The 8 guards cover the right invariants: symlink layout (#1027/#1519), ordering (data file before symlink), and root-owned read-only config (#514).
One housekeeping note: the status: rebase label looks stale now that this has been rebased onto current main. May want to remove it before merge.
LGTM.
, NVIDIA#514) (NVIDIA#1236) ## Summary Reboot of this PR after NVIDIA#1519 merged the actual code fix for issue NVIDIA#1027. This branch is now **test-only** — it adds static regression guards so a future refactor that drops the provisioning steps fails fast in CI, before a docker build runs. The original NVIDIA#1236 carried the code fix alongside tests; since upstream already has the fix from NVIDIA#1519, I rebased to upstream/main and kept only the missing regression coverage. ## What this adds `test/sandbox-provisioning.test.ts` — 8 static assertions on the image-build sources: **NVIDIA#1027 / NVIDIA#1519 (runtime-writable symlinks)** - `Dockerfile.base` creates the `exec-approvals.json` backing file in `.openclaw-data` - `Dockerfile.base` symlinks `.openclaw/exec-approvals.json` → `.openclaw-data/exec-approvals.json` - Same pair for `update-check.json` - Ordering guard: the data file is created before the symlink that points at it **NVIDIA#514 (root-owned read-only config)** - `Dockerfile` keeps `openclaw.json` at mode 0444 (agent cannot tamper with auth token / CORS) - `Dockerfile` keeps `.config-hash` at root:root 0444 (agent cannot forge integrity hash) - `Dockerfile` keeps `.openclaw/` at root:root 0755 (agent cannot replace symlinks) ## Why these tests matter - The existing `test/exec-approvals-path-regression.test.ts` only verifies the path-patching grep logic in `Dockerfile.base`, not the actual symlink/data file creation - The existing `test/e2e-gateway-isolation.sh` catches regressions at image-build time, but requires docker and a full build; static tests fail in seconds on every PR - NVIDIA#1519 was a 6-line code fix with zero test coverage. If a future cleanup of `Dockerfile.base` drops those six lines, the issue returns silently ## Test plan - [x] `npx vitest run test/sandbox-provisioning.test.ts` — 8/8 passing - [x] Rebased onto current `upstream/main` (`d9aced49`) - [x] No code changes; test-only PR Signed-off-by: Tony Luo <xialuo@nvidia.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Tony Luo <xialuo@nvidia.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Aaron Erickson 🦞 <aerickson@nvidia.com>
Summary
Reboot of this PR after #1519 merged the actual code fix for issue #1027. This branch is now test-only — it adds static regression guards so a future refactor that drops the provisioning steps fails fast in CI, before a docker build runs.
The original #1236 carried the code fix alongside tests; since upstream already has the fix from #1519, I rebased to upstream/main and kept only the missing regression coverage.
What this adds
test/sandbox-provisioning.test.ts— 8 static assertions on the image-build sources:#1027 / #1519 (runtime-writable symlinks)
Dockerfile.basecreates theexec-approvals.jsonbacking file in.openclaw-dataDockerfile.basesymlinks.openclaw/exec-approvals.json→.openclaw-data/exec-approvals.jsonupdate-check.json#514 (root-owned read-only config)
Dockerfilekeepsopenclaw.jsonat mode 0444 (agent cannot tamper with auth token / CORS)Dockerfilekeeps.config-hashat root:root 0444 (agent cannot forge integrity hash)Dockerfilekeeps.openclaw/at root:root 0755 (agent cannot replace symlinks)Why these tests matter
test/exec-approvals-path-regression.test.tsonly verifies the path-patching grep logic inDockerfile.base, not the actual symlink/data file creationtest/e2e-gateway-isolation.shcatches regressions at image-build time, but requires docker and a full build; static tests fail in seconds on every PRDockerfile.basedrops those six lines, the issue returns silentlyTest plan
npx vitest run test/sandbox-provisioning.test.ts— 8/8 passingupstream/main(d9aced49)Signed-off-by: Tony Luo xialuo@nvidia.com
🤖 Generated with Claude Code