Skip to content

test(e2e): add non-root sandbox smoke test#3166

Merged
cv merged 6 commits into
mainfrom
test/e2e-non-root-sandbox-smoke-test
May 8, 2026
Merged

test(e2e): add non-root sandbox smoke test#3166
cv merged 6 commits into
mainfrom
test/e2e-non-root-sandbox-smoke-test

Conversation

@hunglp6d

@hunglp6d hunglp6d commented May 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Adds the test-non-root-sandbox-smoke test from #2571 — a PR-gate job that runs the production image under -security-opt no-new-privileges to catch #2472 and #2482 regressions, without OpenShell, NVIDIA_API_KEY, or live inference.

Related Issue

Part of #2571

Changes

  • New test/e2e-non-root-smoke.sh (host-side bash, no openshell/nemoclaw CLI required):
    • Test 1 — entrypoint setup chain completes cleanly under --security-opt no-new-privileges (regression guard for # 2472; passes a true command via the entrypoint's NEMOCLAW_CMD exec path so the gateway-launch branch is bypassed and we don't need the OpenShell-managed runtime).
    • Test 2 — kernel confirms NoNewPrivs=1 inside the container (defends the test itself against silent typos in the docker flag).
  • New job test-non-root-sandbox-smoke in .github/workflows/pr-self-hosted.yaml — linux-amd64-cpu4, timeout-minutes: 5, needs: build-sandbox-images, reuses the existing isolation-image artifact.
  • Expected results:
my-machine@ab1-cdf40-30:~/NemoClaw$ # Run script
bash test/e2e-non-root-smoke.sh
TEST: 1. Entrypoint setup chain completes under --security-opt no-new-privileges
PASS: entrypoint exited 0 under no-new-privileges (#2472 setup chain healthy)
TEST: 2. Kernel confirms NoNewPrivs=1 inside container (defends against silent flag typos)
PASS: kernel confirms NoNewPrivs=1

========================================
  Results: 2 passed, 0 failed
========================================

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Verification

  • npx prek run --all-files passes
  • npm test passes
  • Tests added or updated for new or changed behavior
  • No secrets, API keys, or credentials committed
  • Docs updated for user-facing behavior changes
  • make docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Hung Le hple@nvidia.com

@copy-pr-bot

copy-pr-bot Bot commented May 7, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented May 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR introduces a new smoke test for non-root container execution. A new Bash script validates that the container entrypoint chain executes successfully and the kernel NoNewPrivs flag is properly set when running under the --security-opt no-new-privileges option. A corresponding CI workflow job is added to run these tests in the self-hosted pipeline after sandbox images are built.

Changes

Non-Root Sandbox Smoke Test

Layer / File(s) Summary
Test Script Foundation
test/e2e-non-root-smoke.sh		 Configures Docker image selection, initializes ANSI color helpers for result reporting, sets up pass/fail counters, validates image existence, and defines the run_under_nnp() helper function.
Test Cases
test/e2e-non-root-smoke.sh		 Test 1 verifies the entrypoint chain exits successfully (exit code 0) under no-new-privileges with a true command. Test 2 verifies the kernel NoNewPrivs flag inside the container equals 1.
Test Reporting & Exit Logic
test/e2e-non-root-smoke.sh		 Outputs summary counts of passed and failed tests, then exits with status 1 if any test failed.
CI Workflow Integration
.github/workflows/pr-self-hosted.yaml		 Adds test-non-root-sandbox-smoke job that runs on linux-amd64-cpu4, depends on build-sandbox-images, downloads and loads the isolation-image artifact, and executes the smoke test with NEMOCLAW_TEST_IMAGE=nemoclaw-production.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A sandbox so secure, no privileges new,
The kernel speaks truth—NoNewPrivs rings true!
Entrypoint chains dance in containment so tight,
Tests catch the mischief, keeping the tight—
Smoke clears, all passes, the rabbit takes flight! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: adding a non-root sandbox smoke test for e2e testing.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/e2e-non-root-sandbox-smoke-test

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented May 7, 2026

Copy link
Copy Markdown
Contributor

This repository limits contributors to 10 open pull requests. Please close or merge existing PRs before opening new ones.

@github-actions github-actions Bot closed this May 7, 2026
@hunglp6d hunglp6d reopened this May 7, 2026
@hunglp6d hunglp6d added E2E VRDC Issues and PRs submitted by NVIDIA VRDC test team. labels May 7, 2026
@jyaunches

jyaunches commented May 7, 2026

Copy link
Copy Markdown
Contributor

Thanks @hunglp6d — option 1 is the right call. Land Tests 1–3 now; Test 4 (openclaw tui gateway token) moves to a follow-up that rides with #2485.

Rationale matches yours: main currently has only the build-time clearing half of the gateway-token security refactor; the startup-time generator lives in #2485. Landing Test 4 as a PR gate before #2485 merges would deterministically red every PR in the repo — that's fail-with-future-fix, not all-pass.

I've added a follow-up requirement to #2571 and left a note on #2485 so Test 4 lands as part of that PR's diff. I'll also close my stale #2711 in favor of this one.

This was referenced May 7, 2026
Open
Draft
Closed
@hunglp6d hunglp6d marked this pull request as ready for review May 8, 2026 03:09
coderabbitai[bot]
coderabbitai Bot reviewed May 8, 2026
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/pr-self-hosted.yaml:
- Line 165: Actionlint is flagging the custom self-hosted runner labels (e.g.,
linux-amd64-cpu4 and linux-arm64-cpu4) used in pr-self-hosted.yaml and
nightly-e2e.yaml; add a repository-root actionlint.yaml with a [runner-label]
section that whitelists those labels (include linux-amd64-cpu4 and
linux-arm64-cpu4) so actionlint accepts them, or alternatively change the
affected jobs in pr-self-hosted.yaml and nightly-e2e.yaml to use GitHub-hosted
runner labels instead if you no longer need self-hosted runners.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5d4c90f5-a16b-46ee-b015-f7f5c15ceab3

📥 Commits

Reviewing files that changed from the base of the PR and between 5397b3a and ec88ca5.

📒 Files selected for processing (2)
  • .github/workflows/pr-self-hosted.yaml
  • test/e2e-non-root-smoke.sh

Comment thread .github/workflows/pr-self-hosted.yaml
@hunglp6d hunglp6d requested a review from jyaunches May 8, 2026 03:38
@cv cv added the v0.0.37 label May 8, 2026
@cv cv merged commit 57470be into main May 8, 2026
20 checks passed
@miyoungc miyoungc mentioned this pull request May 8, 2026
Merged
miyoungc added a commit that referenced this pull request May 9, 2026
@miyoungc @cursoragent
docs: prepare 0.0.38 release docs (#3296)
e0ab181 
## Summary
- Bump the docs release metadata to `0.0.38`.
- Document release-prep updates for status policy versions, Local Ollama
validation and cleanup, blueprint policy additions, rebuild backup
handling, and NemoHermes uninstall branding.
- Refresh generated `nemoclaw-user-*` skills from the updated docs.

## Source summary
- #3185 -> `docs/reference/commands.md`: Documents that `nemoclaw <name>
status` displays the gateway active policy version when OpenShell
reports one.
- #3167 -> `docs/reference/commands.md`,
`docs/inference/use-local-inference.md`: Documents uninstall cleanup for
matching Local Ollama auth proxy processes.
- #2737 -> `docs/inference/use-local-inference.md`,
`docs/network-policy/customize-network-policy.md`,
`docs/manage-sandboxes/lifecycle.md`, `docs/reference/commands.md`:
Documents stricter Local Ollama tool-call validation, blueprint policy
additions, and partial rebuild backup handling.
- #3220 -> `docs/reference/commands.md`: Documents NemoHermes-specific
uninstall progress and completion text.
- #3158 -> `.agents/skills/nemoclaw-user-configure-inference/*`:
Refreshes generated user skills from existing
`docs/inference/switch-inference-providers.md` heartbeat documentation.
- #3199 -> `.agents/skills/nemoclaw-user-get-started/SKILL.md`:
Refreshes generated user skills from existing
`docs/get-started/quickstart.md` Model Router wording.

## Skipped
- #3272 and #3268 were already documented by their merged docs updates
on `main`.
- #3154, #3216, #3166, and #3195 have no additional user-facing docs
impact for this release-prep pass.
- No commits matched `docs/.docs-skip`.

## Test plan
- `python3 scripts/docs-to-skills.py docs/ .agents/skills/ --prefix
nemoclaw-user`
- `make docs`
- `npm run build:cli`
- Commit and pre-push hooks: markdownlint, docs-to-skills verification,
gitleaks, commitlint, skills YAML tests, CLI typecheck


Made with [Cursor](https://cursor.com)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Behavior Changes**
* Rebuild now safely handles partial backups, preserving successfully
captured entries while reporting only unarchived paths
* Uninstall for Local Ollama setups now stops proxy processes before
cleanup
* Local Ollama models require stricter tool-call response validation
during onboarding
* Blueprint policy additions enable custom network policy extensions via
`components.policy.additions`
* New `NEMOCLAW_AGENT_HEARTBEAT_EVERY` configuration controls agent
periodic task frequency
  * Status display now shows active policy version when available

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Cursor <cursoragent@cursor.com>
@wscurran wscurran added area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance and removed E2E labels Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@cv cv cv approved these changes

@jyaunches jyaunches Awaiting requested review from jyaunches

Assignees

No one assigned

Labels

area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance VRDC Issues and PRs submitted by NVIDIA VRDC test team.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@hunglp6d @jyaunches @cv @wscurran