fix(rebuild): restore policy presets after sandbox rebuild (#1952)#2026
Conversation
During rebuild, policy presets (telegram, npm, etc.) were lost because they live in the gateway policy engine, not the sandbox filesystem. The rebuild flow (backup → destroy → onboard → restore) recreated the sandbox with the base policy, silently dropping all preset-applied rules. Changes: - sandbox-state.ts: Capture applied policyPresets in RebuildManifest during backup by reading them from the registry - nemoclaw.ts: Add Step 5.5 to rebuild flow that re-applies each saved preset via policies.applyPreset() after state restore - Non-fatal: preset restore failures warn but don't fail the rebuild - Backward compatible: old manifests without policyPresets default to [] Tests: - Unit tests for manifest field, serialization round-trip, backward compat - E2E: extended test-rebuild-openclaw.sh with policy preset apply + verify Fixes: #1952
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughBackup now records applied gateway policy presets in the rebuild manifest; during sandbox rebuild the saved presets are re-applied to the recreated sandbox via the policy engine and per-preset results are logged and surfaced to the user. Changes
Sequence DiagramsequenceDiagram
participant User as User
participant Rebuild as Rebuild Flow
participant Backup as Backup System
participant Registry as Registry
participant Restore as Restore System
participant PolicyEngine as Policy Engine
User->>Rebuild: nemoclaw rebuild --yes
Rebuild->>Backup: backupSandboxState()
Backup->>Registry: read sb.policies
Registry-->>Backup: [policy preset list]
Backup->>Backup: write manifest.policyPresets
Backup-->>Rebuild: backup complete
Rebuild->>Restore: destroy & recreate sandbox
Restore->>Restore: restore workspace files/state
Rebuild->>PolicyEngine: (Step 5.5) for each manifest.policyPresets[]
loop For each preset
Rebuild->>PolicyEngine: applyPreset(sandboxName, presetName)
PolicyEngine-->>Rebuild: success / error
Rebuild->>Rebuild: record result & log
end
Rebuild-->>User: rebuild complete (with preset restore results)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
During rebuild, policy presets (telegram, npm, etc.) were lost because they live in the gateway policy engine, not the sandbox filesystem. The rebuild flow (backup → destroy → onboard → restore) recreated the sandbox with the base policy, silently dropping all preset-applied rules. Changes: - sandbox-state.ts: Capture applied policyPresets in RebuildManifest during backup by reading them from the registry - nemoclaw.ts: Add Step 5.5 to rebuild flow that re-applies each saved preset via policies.applyPreset() after state restore - Non-fatal: preset restore failures warn but don't fail the rebuild - Backward compatible: old manifests without policyPresets default to [] Tests: - Unit tests for manifest field, serialization round-trip, backward compat - E2E: extended test-rebuild-openclaw.sh with policy preset apply + verify - E2E fix: mark preflight/gateway steps complete in session so rebuild resume skips port-8080 preflight check on non-clean environments Fixes: #1952
There was a problem hiding this comment.
🧹 Nitpick comments (1)
test/rebuild-policy-presets.test.ts (1)
133-161: Tests verify expected fallback pattern but don't exercise actual restore code.These tests document the expected
|| []fallback behavior used innemoclaw.ts:1886, which is valuable for regression prevention. However, they test JavaScript semantics rather than the actualsandboxRebuildfunction.Consider adding an integration test that mocks
policies.applyPreset()and verifies it's called with the expected preset names during rebuild. This would provide stronger confidence in the restore path.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/rebuild-policy-presets.test.ts` around lines 133 - 161, Add an integration-style test that invokes sandboxRebuild (the rebuild entrypoint) with a backup/manifest that contains policyPresets = ["telegram","npm"], mock policies.applyPreset to a spy/stub, run sandboxRebuild, and assert policies.applyPreset was called twice with "telegram" and "npm" (and not called when manifest.policyPresets is undefined/null or empty); ensure the test sets up the same || [] fallback scenario and restores the mock after the test so other tests are unaffected.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@test/rebuild-policy-presets.test.ts`:
- Around line 133-161: Add an integration-style test that invokes sandboxRebuild
(the rebuild entrypoint) with a backup/manifest that contains policyPresets =
["telegram","npm"], mock policies.applyPreset to a spy/stub, run sandboxRebuild,
and assert policies.applyPreset was called twice with "telegram" and "npm" (and
not called when manifest.policyPresets is undefined/null or empty); ensure the
test sets up the same || [] fallback scenario and restores the mock after the
test so other tests are unaffected.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 97ccebdf-b2ee-43d8-a482-14f081a66570
📒 Files selected for processing (4)
src/lib/sandbox-state.tssrc/nemoclaw.tstest/e2e/test-rebuild-openclaw.shtest/rebuild-policy-presets.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
test/e2e/test-rebuild-openclaw.sh (1)
174-188:⚠️ Potential issue | 🟠 MajorDon't pre-seed the registry with the presets under test.
backupSandboxState()readsregistry.getSandbox(...).policies, but Lines 176-183 write['npm', 'pypi']intosandboxes.jsonbefore Phase 4.5 runs. That means the rebuild manifest and post-rebuild registry checks can still pass even ifpolicies.applyPreset()stops persisting presets to the registry. Start with an empty/missingpolicieslist here, then let Phase 4.5 prove the preset application populated the registry.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/test-rebuild-openclaw.sh` around lines 174 - 188, The test pre-seeds the sandbox registry with presets causing backupSandboxState() to read policies and masking failures in policies.applyPreset(); modify the JSON written in the inline Python so the sandbox entry does not include a pre-populated "policies" value (either remove the "policies" key entirely or set it to an empty list) so Phase 4.5 must populate the registry itself; leave other keys like "policyTier" and "agentVersion" unchanged and keep defaultSandbox as-is so subsequent checks validate that policies.applyPreset() persisted the presets.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/e2e/test-rebuild-openclaw.sh`:
- Around line 420-438: The test currently only verifies policyPresets is
non-empty; update the check that reads MANIFEST_PRESETS (from
rebuild-manifest.json) to assert it contains exactly the two expected preset
names (not just non-empty). Parse MANIFEST_PRESETS into a list and compare
against the expected set (order-independent) and only call pass "Backup manifest
contains policyPresets: ..." when the sets match; otherwise call fail with a
clear message listing the missing/extra presets so partial or incorrect presets
are caught.
---
Outside diff comments:
In `@test/e2e/test-rebuild-openclaw.sh`:
- Around line 174-188: The test pre-seeds the sandbox registry with presets
causing backupSandboxState() to read policies and masking failures in
policies.applyPreset(); modify the JSON written in the inline Python so the
sandbox entry does not include a pre-populated "policies" value (either remove
the "policies" key entirely or set it to an empty list) so Phase 4.5 must
populate the registry itself; leave other keys like "policyTier" and
"agentVersion" unchanged and keep defaultSandbox as-is so subsequent checks
validate that policies.applyPreset() persisted the presets.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: cdf3c3a0-3a82-434f-b233-cd2ac316a073
📒 Files selected for processing (1)
test/e2e/test-rebuild-openclaw.sh
Assert both npm and pypi are present in the backup manifest's policyPresets field, not just that it is non-empty. Catches partial backup regressions.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
test/e2e/test-rebuild-openclaw.sh (1)
434-439:⚠️ Potential issue | 🟡 MinorManifest preset assertion still isn’t exact-set validation.
This still checks containment only; extra or malformed preset names can pass. The assertion should compare actual vs expected sets and report missing/extra entries explicitly.
Use exact-set comparison with clear mismatch output
- if echo "${MANIFEST_PRESETS}" | grep -q "npm" \ - && echo "${MANIFEST_PRESETS}" | grep -q "pypi"; then - pass "Backup manifest contains policyPresets: ${MANIFEST_PRESETS}" - else - fail "Backup manifest missing expected policyPresets (npm,pypi): got '${MANIFEST_PRESETS}' — issue `#1952`" - fi + MANIFEST_SET_CHECK=$(python3 -c " +expected = {'npm', 'pypi'} +raw = '${MANIFEST_PRESETS}' +actual = {x for x in raw.split(',') if x and x not in {'NONE', 'error'} and not x.startswith('ERROR:')} +missing = sorted(expected - actual) +extra = sorted(actual - expected) +print('OK' if not missing and not extra else f'missing={missing}, extra={extra}') +") + if [ \"${MANIFEST_SET_CHECK}\" = \"OK\" ]; then + pass \"Backup manifest contains expected policyPresets: ${MANIFEST_PRESETS}\" + else + fail \"Backup manifest policyPresets mismatch (${MANIFEST_SET_CHECK}): got '${MANIFEST_PRESETS}' — issue `#1952`\" + fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/test-rebuild-openclaw.sh` around lines 434 - 439, Replace the containment checks on MANIFEST_PRESETS with an exact-set comparison: parse MANIFEST_PRESETS into a normalized sorted list (split on commas/whitespace), build the expected set ["npm","pypi"], compare both sets for equality, and call pass when equal; on mismatch call fail and include explicit missing entries (expected but not present) and extra entries (present but not expected). Update the conditional around MANIFEST_PRESETS and the pass/fail branches (the logic currently using grep -q) to perform this set diff so test/e2e/test-rebuild-openclaw.sh reports precise missing/extra preset names.
🧹 Nitpick comments (1)
test/e2e/test-rebuild-openclaw.sh (1)
396-405: Registry preset survival check is substring-based and can false-pass.Line 396 and Line 401 grep for
"npm"/"pypi"in a CSV string. Values likenpm-mirrorwould pass. Parse as a set and compare exact expected values.Tighten to an order-independent exact-set assertion
-if echo "${POST_REBUILD_PRESETS}" | grep -q "npm"; then - pass "npm preset survived rebuild (in registry)" -else - fail "npm preset LOST after rebuild — issue `#1952`" -fi -if echo "${POST_REBUILD_PRESETS}" | grep -q "pypi"; then - pass "pypi preset survived rebuild (in registry)" -else - fail "pypi preset LOST after rebuild — issue `#1952`" -fi +REGISTRY_SET_CHECK=$(python3 -c " +expected = {'npm', 'pypi'} +actual = {x for x in '${POST_REBUILD_PRESETS}'.split(',') if x and x != 'error'} +missing = sorted(expected - actual) +extra = sorted(actual - expected) +print('OK' if not missing and not extra else f'missing={missing}, extra={extra}') +") +if [ "${REGISTRY_SET_CHECK}" = "OK" ]; then + pass "Registry presets survived rebuild exactly: ${POST_REBUILD_PRESETS}" +else + fail "Registry presets mismatch after rebuild (${REGISTRY_SET_CHECK}) — issue `#1952`" +fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/test-rebuild-openclaw.sh` around lines 396 - 405, The current checks use substring grep on POST_REBUILD_PRESETS which can false-pass (e.g. "npm-mirror"); change the assertion to parse POST_REBUILD_PRESETS as a CSV, trim each entry, and perform an order-independent exact membership test against the expected set {"npm","pypi"} (e.g. split on ',', trim whitespace, sort/uniq entries and compare to the sorted expected list or check each exact element exists in the parsed array). Update the checks around POST_REBUILD_PRESETS (the grep tests that pass/fail "npm" and "pypi") to use this exact-set comparison instead of simple grep.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@test/e2e/test-rebuild-openclaw.sh`:
- Around line 434-439: Replace the containment checks on MANIFEST_PRESETS with
an exact-set comparison: parse MANIFEST_PRESETS into a normalized sorted list
(split on commas/whitespace), build the expected set ["npm","pypi"], compare
both sets for equality, and call pass when equal; on mismatch call fail and
include explicit missing entries (expected but not present) and extra entries
(present but not expected). Update the conditional around MANIFEST_PRESETS and
the pass/fail branches (the logic currently using grep -q) to perform this set
diff so test/e2e/test-rebuild-openclaw.sh reports precise missing/extra preset
names.
---
Nitpick comments:
In `@test/e2e/test-rebuild-openclaw.sh`:
- Around line 396-405: The current checks use substring grep on
POST_REBUILD_PRESETS which can false-pass (e.g. "npm-mirror"); change the
assertion to parse POST_REBUILD_PRESETS as a CSV, trim each entry, and perform
an order-independent exact membership test against the expected set
{"npm","pypi"} (e.g. split on ',', trim whitespace, sort/uniq entries and
compare to the sorted expected list or check each exact element exists in the
parsed array). Update the checks around POST_REBUILD_PRESETS (the grep tests
that pass/fail "npm" and "pypi") to use this exact-set comparison instead of
simple grep.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: a18490a5-5481-4c42-a825-2295d4e0ccd1
📒 Files selected for processing (1)
test/e2e/test-rebuild-openclaw.sh
|
✨ Possibly related open issues: |
ericksoa
left a comment
There was a problem hiding this comment.
The core fix is correct and well-designed — capturing presets during backup and re-applying after restore is the right approach. The E2E test coverage is solid.
However, this PR cannot merge as-is:
Blocker: Commits authored as "Test User"
All three author commits are signed as Test User <test@example.com>. Commits must use your real git identity. Please rebase with:
git config user.name "J. Yaunches"
git config user.email "jyaunches@nvidia.com"Then rewrite the commits:
git rebase -i main --exec 'git commit --amend --author="J. Yaunches <jyaunches@nvidia.com>" --no-edit'
git push --force-with-leaseMinor feedback
- Unit tests (
test/rebuild-policy-presets.test.ts): Most tests assert basic JS behavior (object property access, JSON round-trip,|| []fallback) rather than exercising the actualbackupSandboxState()or rebuild restore functions. The E2E tests provide the real coverage, so this is acceptable but worth noting. @ts-nocheckon the test file disables all TS checking — consider removing it and adding targeted type assertions instead.
Once the commit identity is fixed, this is ready to approve.
Withdrawing — the commit identity concern was incorrect.
ericksoa
left a comment
There was a problem hiding this comment.
LGTM. The core fix is correct — capturing presets during backup and re-applying after restore is the right approach.
backupSandboxState()cleanly reads presets from existing registry data- Rebuild Step 5.5 is non-fatal: failures warn but don't abort, with manual re-apply guidance
- Backward compatible: old manifests without
policyPresetsdefault to[]via|| [] - E2E tests are solid — apply npm/pypi before rebuild, verify they survive in registry, gateway policy, and backup manifest
Minor notes (non-blocking):
- Unit tests are mostly shallow (test JS object/JSON behavior rather than actual functions), but the E2E coverage compensates
@ts-nocheckon the test file — could be removed with targeted type assertions
## Summary Catches up the user-facing reference and troubleshooting docs with the CLI and policy behavior changes that landed in v0.0.21. Drafted via the `nemoclaw-contributor-update-docs` skill against commits in `v0.0.20..v0.0.21`, filtered through `docs/.docs-skip`. ## Changes - **`docs/reference/commands.md`** - `nemoclaw list`: session indicator (●) for connected sandboxes (#2117). - `nemoclaw <name> connect`: active-session note; auto-recovery from SSH identity drift after a host reboot (#2117, #2064). - `nemoclaw <name> status`: three-state Inference line (`healthy` / `unreachable` / `not probed`) covering both local and remote providers; new `Connected` line (#2002, #2117). - `nemoclaw <name> destroy` and `rebuild`: active-session warning with second confirm; rebuild reapplies policy presets to the recreated sandbox (#2117, #2026). - `nemoclaw <name> policy-add` and `policy-remove`: positional preset argument and non-interactive flow via `--yes`/`--force`/`NEMOCLAW_NON_INTERACTIVE=1` (#2070). - `nemoclaw <name> policy-list`: registry-vs-gateway desync detection (#2089). - **`docs/reference/troubleshooting.md`** - `Reconnect after a host reboot`: now reflects automatic stale `known_hosts` pruning on `connect` (#2064). - `Running multiple sandboxes simultaneously`: onboard's forward-port collision guard (#2086). - New section: `openclaw config set` or `unset` is blocked inside the sandbox (#2081). - **`docs/network-policy/customize-network-policy.md`**: non-interactive `policy-add`/`policy-remove` form; preset preservation across rebuild (#2070, #2026). - **`docs/inference/use-local-inference.md`**: NIM section now covers the NGC API key prompt with masked input and `docker login nvcr.io --password-stdin` behavior (#2043). - **Generated skills regenerated** to pick up the source changes (`.agents/skills/nemoclaw-user-reference/references/{commands,troubleshooting}.md`, plus minor heading-flow deltas elsewhere). The pre-commit `Regenerate agent skills from docs` hook ran and confirmed source ↔ generated parity. Commits skipped per `docs/.docs-skip` or no doc impact: `bbbaa0fb` (skip-features), `7cb482cb` (skip-features), `8dee23fd` (skip-terms), plus the usual CI / test / refactor / install-plumbing churn. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [x] Doc only (includes code sample changes) ## Verification - [x] `npx prek run --all-files` passes for the modified files (the one failing test, `test/cli.test.ts > unknown command exits 1`, also fails on `origin/main` and is unrelated to these markdown-only changes) - [ ] `npm test` passes — skipped; same pre-existing CLI-dispatch test failure unrelated to docs - [ ] Tests added or updated for new or changed behavior — n/a, doc-only - [x] No secrets, API keys, or credentials committed - [x] Docs updated for user-facing behavior changes - [ ] `make docs` builds without warnings (doc changes only) — not run locally - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) — n/a, no new pages ## AI Disclosure - [x] AI-assisted — tool: Claude Code --- Signed-off-by: Miyoung Choi <miyoungc@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Multi-session SSH connections with concurrent session support. * Three-state inference health reporting (healthy/unreachable/not probed) across all providers. * Automatic SSH host key rotation detection and recovery. * Non-interactive policy preset management via positional arguments. * Session indicators in sandbox list view. * **Bug Fixes** * Protected destructive operations with active-session warnings. * Policy presets now preserved during sandbox rebuilds. * **Documentation** * NGC registry authentication requirements for container images. * Multi-sandbox onboarding and reconnection guidance. * Troubleshooting updates for port conflicts and SSH issues. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
After
nemoclaw <name> rebuild --yes, previously applied network policy presets (e.g., telegram, npm) were lost because they live in the gateway policy engine, not the sandbox filesystem. The rebuild flow recreated the sandbox with the base policy, silently dropping all preset-applied rules. This caused messaging bridges to fail with 403 Forbidden.Related Issue
Fixes #1952
Changes
src/lib/sandbox-state.ts: AddedpolicyPresetsfield toRebuildManifestinterface. DuringbackupSandboxState(), applied presets are read from the registry and saved in the manifest.src/nemoclaw.ts: Added Step 5.5 to the rebuild flow between state restore and post-restore migration. Re-applies each saved preset viapolicies.applyPreset(). Non-fatal: failures warn but do not fail the rebuild.test/rebuild-policy-presets.test.ts: Unit tests for manifest field, JSON serialization round-trip, backward compatibility with older manifests.test/e2e/test-rebuild-openclaw.sh: Extended with Phase 4.5 (apply npm/pypi presets before rebuild) and Phase 7b (verify presets survived rebuild in registry, gateway policy, and backup manifest).Backward compatible: old manifests without
policyPresetsdefault to[]via|| [].Type of Change
Verification
npx prek run --all-filespassesnpm testpasses (our changes: 9/9 new tests, 79/79 policy tests, 142/142 relevant tests)make docsbuilds without warnings (doc changes only)AI Disclosure
Signed-off-by: J. Yaunches jyaunches@nvidia.com
Summary by CodeRabbit
New Features
Tests