test(e2e): add Brev launchable install-flow smoke test#2677
Conversation
Add test/e2e/test-launchable-smoke.sh — validates the community install path (scripts/brev-launchable-ci-cpu.sh) end-to-end on ubuntu-latest: Phase 2: Run brev-launchable-ci-cpu.sh with NEMOCLAW_REF=current branch Phase 3: Verify artifacts (nemoclaw, openshell, Node ≥22, Docker, sentinel) Phase 4: nemoclaw onboard --non-interactive (cloud provider) Phase 5: Sandbox health (list, status, inference config, gateway) Phase 6: Live inference (direct API, routing via inference.local, openclaw agent) Phase 7: Destroy + cleanup Wire launchable-smoke-e2e into .github/workflows/nightly-e2e.yaml: - runs-on: ubuntu-latest, timeout: 30min - NVIDIA_API_KEY secret, SKIP_DOCKER_PULL=1 - Upload install/onboard/test logs as artifacts on failure - Added to notify-on-failure needs list Key insight: brev-launchable-ci-cpu.sh has zero Brev dependencies — it is a generic Ubuntu bootstrap script that runs on GitHub-hosted runners with no BREV_API_TOKEN needed. Closes #2599
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a nightly GitHub Actions job and a new end-to-end Bash smoke test to validate the Brev Launchable community install flow; the job runs on Changes
Sequence Diagram(s)sequenceDiagram
participant Runner as CI Runner
participant Installer as launch-plugin.sh / test script
participant Docker as Docker daemon
participant Sandbox as Brev Sandbox / Gateway
participant OpenShell as OpenShell / CLI
participant NVIDIA as integrate.api.nvidia.com
Runner->>Installer: set envs & execute `test-launchable-smoke.sh`
Installer->>Docker: ensure Docker running, run installer, start gateway container
Docker-->>Installer: container started / health OK
Installer->>Sandbox: run `nemoclaw onboard --non-interactive`
Sandbox->>OpenShell: expose CLI and inference endpoint
OpenShell->>NVIDIA: call NVIDIA endpoint (expect "PONG")
Runner->>Sandbox: SSH/curl https://inference.local (retries)
Runner->>Sandbox: run `openclaw agent` and assert result == 42
Runner->>Installer: destroy sandbox/gateway (unless KEEP_SANDBOX)
Installer-->>Runner: upload logs on failure
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Review rate limit: 7/10 reviews remaining, refill in 13 minutes and 54 seconds. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
test/e2e/test-launchable-smoke.sh (1)
444-446: Remove deadTIMEOUT_CMDassignments (SC2034).
TIMEOUT_CMDis assigned on Line 444-446 but never used. This is dead code and keeps ShellCheck noisy.Suggested fix
-TIMEOUT_CMD="" -command -v timeout >/dev/null 2>&1 && TIMEOUT_CMD="timeout" -command -v gtimeout >/dev/null 2>&1 && TIMEOUT_CMD="gtimeout"As per coding guidelines,
**/*.shfiles must include ShellCheck compliance.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/test-launchable-smoke.sh` around lines 444 - 446, The variable TIMEOUT_CMD is assigned via the command -v checks but never used, causing dead code and ShellCheck SC2034; remove the three lines that declare/assign TIMEOUT_CMD (the variable and the two command -v checks) from the script, or if a timeout wrapper was intended, replace the dead assignments by actually using TIMEOUT_CMD where timeouts are invoked (e.g., use "$TIMEOUT_CMD" before commands) or export a single explicit timeout command; specifically locate the occurrences of the symbol TIMEOUT_CMD and delete the assignment block if unused, or wire it into timeout invocations to eliminate the dead-variable warning.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Line 223: SC2024 is triggered by redirecting output outside sudo for the
backgrounded invocation of brev-launchable-ci-cpu.sh; fix by running the
redirection inside the elevated shell so the file is created/owned under the
sudo context, e.g. replace the current command with a sudo -E bash -c "exec
\"$REPO/scripts/brev-launchable-ci-cpu.sh\" >\"$INSTALL_LOG\" 2>&1 &" form
(ensuring INSTALL_LOG is quoted), or if the original behavior of creating a
runner-owned log file is intentional, add a targeted ShellCheck suppression
comment (# shellcheck disable=SC2024) immediately above that line with a short
rationale referencing SC2024 and INSTALL_LOG.
- Around line 157-164: The script currently always calls pass "Pre-cleanup
complete (clone dir pre-seeded)" even if the preceding setup steps fail; make
the clone/pre-seed commands fail fast and only call pass on success: remove the
tolerant "|| true" on the git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD
command (or replace it with explicit error handling), ensure the git clone and
git -C ... remote set-url origin commands are checked for non-zero exit codes
(or enable a shell-wide "set -e" at the top of the script), and if any of those
commands fail invoke the script's existing failure handler (or exit non-zero)
instead of proceeding to call pass "Pre-cleanup complete (clone dir
pre-seeded)"; reference the NEMOCLAW_CLONE_DIR variable and the pass invocation
to locate the lines to change.
---
Nitpick comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 444-446: The variable TIMEOUT_CMD is assigned via the command -v
checks but never used, causing dead code and ShellCheck SC2034; remove the three
lines that declare/assign TIMEOUT_CMD (the variable and the two command -v
checks) from the script, or if a timeout wrapper was intended, replace the dead
assignments by actually using TIMEOUT_CMD where timeouts are invoked (e.g., use
"$TIMEOUT_CMD" before commands) or export a single explicit timeout command;
specifically locate the occurrences of the symbol TIMEOUT_CMD and delete the
assignment block if unused, or wire it into timeout invocations to eliminate the
dead-variable warning.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: b3e45d3d-7c18-475b-8ce8-9da6768fab3e
📒 Files selected for processing (2)
.github/workflows/nightly-e2e.yamltest/e2e/test-launchable-smoke.sh
| git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR" | ||
| # Ensure a "main" branch exists at the current commit for the script's | ||
| # `git fetch origin main && git checkout main` to succeed. Point origin | ||
| # at the clone itself so fetch resolves locally (the CI checkout may be | ||
| # in detached HEAD and lack a "main" branch). | ||
| git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true | ||
| git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR" | ||
| pass "Pre-cleanup complete (clone dir pre-seeded)" |
There was a problem hiding this comment.
Fail fast on clone pre-seed errors before marking PASS.
Line 157, Line 162, and Line 163 are all critical setup operations, but Line 164 always records success even if one of those commands fails. That can hide root-cause install failures and make the smoke test report misleading.
Suggested fix
git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
+if [ $? -ne 0 ]; then
+ fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR"
+ exit 1
+fi
# Ensure a "main" branch exists at the current commit for the script's
# `git fetch origin main && git checkout main` to succeed. Point origin
# at the clone itself so fetch resolves locally (the CI checkout may be
# in detached HEAD and lack a "main" branch).
git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
-git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"
+if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then
+ fail "Failed to repoint origin for pre-seeded clone"
+ exit 1
+fi
pass "Pre-cleanup complete (clone dir pre-seeded)"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR" | |
| # Ensure a "main" branch exists at the current commit for the script's | |
| # `git fetch origin main && git checkout main` to succeed. Point origin | |
| # at the clone itself so fetch resolves locally (the CI checkout may be | |
| # in detached HEAD and lack a "main" branch). | |
| git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true | |
| git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR" | |
| pass "Pre-cleanup complete (clone dir pre-seeded)" | |
| git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR" | |
| if [ $? -ne 0 ]; then | |
| fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR" | |
| exit 1 | |
| fi | |
| # Ensure a "main" branch exists at the current commit for the script's | |
| # `git fetch origin main && git checkout main` to succeed. Point origin | |
| # at the clone itself so fetch resolves locally (the CI checkout may be | |
| # in detached HEAD and lack a "main" branch). | |
| git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true | |
| if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then | |
| fail "Failed to repoint origin for pre-seeded clone" | |
| exit 1 | |
| fi | |
| pass "Pre-cleanup complete (clone dir pre-seeded)" |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/e2e/test-launchable-smoke.sh` around lines 157 - 164, The script
currently always calls pass "Pre-cleanup complete (clone dir pre-seeded)" even
if the preceding setup steps fail; make the clone/pre-seed commands fail fast
and only call pass on success: remove the tolerant "|| true" on the git -C
"$NEMOCLAW_CLONE_DIR" checkout -B main HEAD command (or replace it with explicit
error handling), ensure the git clone and git -C ... remote set-url origin
commands are checked for non-zero exit codes (or enable a shell-wide "set -e" at
the top of the script), and if any of those commands fail invoke the script's
existing failure handler (or exit non-zero) instead of proceeding to call pass
"Pre-cleanup complete (clone dir pre-seeded)"; reference the NEMOCLAW_CLONE_DIR
variable and the pass invocation to locate the lines to change.
Rename the existing Brev E2E workflow to clearly communicate its purpose: validating a git branch by installing from source on a clean machine. This distinguishes it from the new e2e-launchable-smoke job (added in the prior commit) which validates the community install path that real Brev Launchable users run. Changes: - Rename .github/workflows/e2e-brev.yaml → e2e-branch-validation.yaml - Update workflow name, job name, concurrency group, artifact names - Add expanded header comments explaining purpose, how/when to use - Update vitest project name: e2e-brev → e2e-branch-validation - Update brev-e2e.test.ts doc comment with clear purpose + cross-ref - Update AGENTS.md reference Part of #2599
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
.github/workflows/e2e-branch-validation.yaml (1)
120-121: Concurrency fallback currently disables dedupe for non-PR runs.When
inputs.pr_numberis empty, usinggithub.run_idmakes the group unique every run, socancel-in-progressnever helps for branch-basedworkflow_callruns.Suggested patch
-concurrency: - group: e2e-branch-validation-${{ inputs.pr_number || github.run_id }} +concurrency: + group: e2e-branch-validation-${{ inputs.pr_number || inputs.branch || github.ref_name || github.run_id }} cancel-in-progress: true🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/e2e-branch-validation.yaml around lines 120 - 121, The concurrency group uses a per-run unique fallback (github.run_id) so cancel-in-progress never dedupes non-PR workflow_call runs; change the expression used in the concurrency group from e2e-branch-validation-${{ inputs.pr_number || github.run_id }} to a stable branch/ref fallback such as e2e-branch-validation-${{ inputs.pr_number || github.ref }} (or github.ref_name) so branch-based runs share the same group; update the group string accordingly where "group: e2e-branch-validation-${{ inputs.pr_number || github.run_id }}" appears and keep "cancel-in-progress: true".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/e2e-branch-validation.yaml:
- Around line 25-27: Update the header comment that currently references
"e2e-launchable-smoke" to the correct job identifier "launchable-smoke-e2e" so
the note matches this PR's context; search for the string "e2e-launchable-smoke"
in the header comment near the top of
.github/workflows/e2e-branch-validation.yaml and replace it with
"launchable-smoke-e2e" to avoid misrouting maintainers.
---
Nitpick comments:
In @.github/workflows/e2e-branch-validation.yaml:
- Around line 120-121: The concurrency group uses a per-run unique fallback
(github.run_id) so cancel-in-progress never dedupes non-PR workflow_call runs;
change the expression used in the concurrency group from
e2e-branch-validation-${{ inputs.pr_number || github.run_id }} to a stable
branch/ref fallback such as e2e-branch-validation-${{ inputs.pr_number ||
github.ref }} (or github.ref_name) so branch-based runs share the same group;
update the group string accordingly where "group: e2e-branch-validation-${{
inputs.pr_number || github.run_id }}" appears and keep "cancel-in-progress:
true".
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5726259b-4065-46cc-bad7-496829a341d3
📒 Files selected for processing (4)
.github/workflows/e2e-branch-validation.yamlAGENTS.mdtest/e2e/brev-e2e.test.tsvitest.config.ts
✅ Files skipped from review due to trivial changes (3)
- AGENTS.md
- vitest.config.ts
- test/e2e/brev-e2e.test.ts
| # NOTE: This does NOT test the community install path (launch-plugin.sh). | ||
| # For validating what Brev Launchable users actually get, see the | ||
| # e2e-launchable-smoke job in nightly-e2e.yaml. |
There was a problem hiding this comment.
Fix stale launchable job identifier in the header note.
The comment references e2e-launchable-smoke, but this PR context uses launchable-smoke-e2e. Keeping this exact avoids misrouting maintainers.
Suggested patch
-# e2e-launchable-smoke job in nightly-e2e.yaml.
+# launchable-smoke-e2e job in nightly-e2e.yaml.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # NOTE: This does NOT test the community install path (launch-plugin.sh). | |
| # For validating what Brev Launchable users actually get, see the | |
| # e2e-launchable-smoke job in nightly-e2e.yaml. | |
| # NOTE: This does NOT test the community install path (launch-plugin.sh). | |
| # For validating what Brev Launchable users actually get, see the | |
| # launchable-smoke-e2e job in nightly-e2e.yaml. |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/e2e-branch-validation.yaml around lines 25 - 27, Update
the header comment that currently references "e2e-launchable-smoke" to the
correct job identifier "launchable-smoke-e2e" so the note matches this PR's
context; search for the string "e2e-launchable-smoke" in the header comment near
the top of .github/workflows/e2e-branch-validation.yaml and replace it with
"launchable-smoke-e2e" to avoid misrouting maintainers.
Resolve conflict with credential-migration-e2e comment block from main. Add selective dispatch guard to launchable-smoke-e2e to match the pattern from PR #2637. Add launchable-smoke-e2e to the valid jobs input list.
Selective E2E Results — ✅ All requested jobs passedRun: 25125393577
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/nightly-e2e.yaml:
- Around line 1288-1294: The new job launchable-smoke-e2e must be added to the
dependency list for the report-to-pr job so the PR summary waits for it; update
the report-to-pr job's needs array (referencing the job name report-to-pr and
the new launchable-smoke-e2e job) to include "launchable-smoke-e2e" alongside
the existing needs entries so report-to-pr runs only after that job completes.
- Around line 1288-1334: Add a new path_instructions mapping in .coderabbit.yaml
associating the test file test/e2e/test-launchable-smoke.sh with the job name
launchable-smoke-e2e; locate the existing path_instructions block and add an
entry following the same pattern as other E2E jobs (map the file path to the job
identifier "launchable-smoke-e2e") so the notify-on-failure coverage includes
this new job.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: b97e7663-db60-4f8c-8115-2016615e1fa4
📒 Files selected for processing (3)
.github/workflows/nightly-e2e.yamlAGENTS.mdvitest.config.ts
✅ Files skipped from review due to trivial changes (2)
- AGENTS.md
- vitest.config.ts
| launchable-smoke-e2e: | ||
| if: >- | ||
| github.repository == 'NVIDIA/NemoClaw' && | ||
| (github.event_name != 'workflow_dispatch' || | ||
| inputs.jobs == '' || | ||
| contains(format(',{0},', inputs.jobs), ',launchable-smoke-e2e,')) | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
Add launchable-smoke-e2e to report-to-pr.needs to avoid incomplete PR results.
Line 1288 introduces a new runnable job, but report-to-pr (Lines 1531-1555) does not depend on it. That can post a PR summary before this job finishes and omit its status.
🔧 Proposed fix
report-to-pr:
runs-on: ubuntu-latest
needs:
[
cloud-e2e,
cloud-onboard-e2e,
cloud-inference-e2e,
skill-agent-e2e,
docs-validation-e2e,
messaging-providers-e2e,
token-rotation-e2e,
sandbox-survival-e2e,
hermes-e2e,
skip-permissions-e2e,
sandbox-operations-e2e,
inference-routing-e2e,
network-policy-e2e,
deployment-services-e2e,
diagnostics-e2e,
snapshot-commands-e2e,
shields-config-e2e,
rebuild-openclaw-e2e,
upgrade-stale-sandbox-e2e,
rebuild-hermes-e2e,
overlayfs-autofix-e2e,
+ launchable-smoke-e2e,
gpu-e2e,
]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| launchable-smoke-e2e: | |
| if: >- | |
| github.repository == 'NVIDIA/NemoClaw' && | |
| (github.event_name != 'workflow_dispatch' || | |
| inputs.jobs == '' || | |
| contains(format(',{0},', inputs.jobs), ',launchable-smoke-e2e,')) | |
| runs-on: ubuntu-latest | |
| report-to-pr: | |
| runs-on: ubuntu-latest | |
| needs: | |
| [ | |
| cloud-e2e, | |
| cloud-onboard-e2e, | |
| cloud-inference-e2e, | |
| skill-agent-e2e, | |
| docs-validation-e2e, | |
| messaging-providers-e2e, | |
| token-rotation-e2e, | |
| sandbox-survival-e2e, | |
| hermes-e2e, | |
| skip-permissions-e2e, | |
| sandbox-operations-e2e, | |
| inference-routing-e2e, | |
| network-policy-e2e, | |
| deployment-services-e2e, | |
| diagnostics-e2e, | |
| snapshot-commands-e2e, | |
| shields-config-e2e, | |
| rebuild-openclaw-e2e, | |
| upgrade-stale-sandbox-e2e, | |
| rebuild-hermes-e2e, | |
| overlayfs-autofix-e2e, | |
| launchable-smoke-e2e, | |
| gpu-e2e, | |
| ] |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/nightly-e2e.yaml around lines 1288 - 1294, The new job
launchable-smoke-e2e must be added to the dependency list for the report-to-pr
job so the PR summary waits for it; update the report-to-pr job's needs array
(referencing the job name report-to-pr and the new launchable-smoke-e2e job) to
include "launchable-smoke-e2e" alongside the existing needs entries so
report-to-pr runs only after that job completes.
| launchable-smoke-e2e: | ||
| if: >- | ||
| github.repository == 'NVIDIA/NemoClaw' && | ||
| (github.event_name != 'workflow_dispatch' || | ||
| inputs.jobs == '' || | ||
| contains(format(',{0},', inputs.jobs), ',launchable-smoke-e2e,')) | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - name: Run launchable install-flow smoke test | ||
| env: | ||
| NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }} | ||
| NEMOCLAW_NON_INTERACTIVE: "1" | ||
| NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE: "1" | ||
| NEMOCLAW_SANDBOX_NAME: "e2e-launchable" | ||
| NEMOCLAW_RECREATE_SANDBOX: "1" | ||
| SKIP_DOCKER_PULL: "1" | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| run: bash test/e2e/test-launchable-smoke.sh | ||
|
|
||
| - name: Upload install log on failure | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: launchable-smoke-install-log | ||
| path: /tmp/nemoclaw-launchable-install.log | ||
| if-no-files-found: ignore | ||
|
|
||
| - name: Upload onboard log on failure | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: launchable-smoke-onboard-log | ||
| path: /tmp/nemoclaw-launchable-onboard.log | ||
| if-no-files-found: ignore | ||
|
|
||
| - name: Upload test log on failure | ||
| if: failure() | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: launchable-smoke-test-log | ||
| path: /tmp/nemoclaw-launchable-test.log | ||
| if-no-files-found: ignore | ||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
cfg="$(fd -a '^\.coderabbit\.yaml$' | head -n1 || true)"
if [ -z "$cfg" ]; then
echo "ERROR: .coderabbit.yaml not found"
exit 1
fi
echo "== Checking .coderabbit.yaml for launchable smoke coverage =="
rg -n -C3 'path_instructions|launchable-smoke-e2e|test/e2e/test-launchable-smoke.sh|brev-launchable-ci-cpu.sh' "$cfg"
echo "== Checking coverage validator test references =="
testfile="$(fd -a '^validate-e2e-coverage\.test\.ts$' | head -n1 || true)"
if [ -n "$testfile" ]; then
rg -n -C3 'launchable-smoke-e2e|test-launchable-smoke' "$testfile"
else
echo "WARN: validate-e2e-coverage.test.ts not found"
fiRepository: NVIDIA/NemoClaw
Length of output: 319
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find .coderabbit.yaml in root with proper hidden file handling
cfg="$(find . -maxdepth 1 -name '.coderabbit.yaml' | head -n1 || true)"
if [ -z "$cfg" ]; then
echo "ERROR: .coderabbit.yaml not found in repository root"
ls -la . | head -20
exit 1
fi
echo "== .coderabbit.yaml found at: $cfg =="
echo ""
echo "== Searching for launchable-smoke-e2e coverage in .coderabbit.yaml =="
rg 'launchable-smoke-e2e' "$cfg" || echo "WARN: launchable-smoke-e2e not found in .coderabbit.yaml"
echo ""
echo "== Checking full path_instructions section for E2E jobs =="
rg -A 200 'path_instructions:' "$cfg" | head -100
echo ""
echo "== Checking if test script exists =="
test_script="test/e2e/test-launchable-smoke.sh"
if [ -f "$test_script" ]; then
echo "✓ Test script exists: $test_script"
else
echo "✗ Test script NOT found: $test_script"
fi
echo ""
echo "== Checking validate-e2e-coverage.test.ts =="
testfile="$(find . -name 'validate-e2e-coverage.test.ts' | head -n1 || true)"
if [ -n "$testfile" ]; then
echo "Found: $testfile"
rg 'launchable-smoke' "$testfile" || echo "WARN: launchable-smoke reference not found in test file"
else
echo "WARN: validate-e2e-coverage.test.ts not found"
fiRepository: NVIDIA/NemoClaw
Length of output: 5756
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Full validate-e2e-coverage.test.ts content =="
cat test/validate-e2e-coverage.test.ts
echo ""
echo "== Search for other E2E job entries in .coderabbit.yaml =="
rg -n 'e2e|smoke|test/' .coderabbit.yaml | head -50
echo ""
echo "== Check for notify-on-failure in .coderabbit.yaml =="
rg -B 5 -A 20 'notify-on-failure' .coderabbit.yaml || echo "WARN: notify-on-failure not found"Repository: NVIDIA/NemoClaw
Length of output: 13179
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Checking notify-on-failure in nightly-e2e.yaml =="
rg -B 3 -A 30 'notify-on-failure:' .github/workflows/nightly-e2e.yaml | head -80Repository: NVIDIA/NemoClaw
Length of output: 1063
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Check if launchable-smoke-e2e appears in nightly-e2e.yaml =="
rg 'launchable-smoke-e2e' .github/workflows/nightly-e2e.yaml
echo ""
echo "== Full notify-on-failure needs list =="
rg -A 50 'notify-on-failure:' .github/workflows/nightly-e2e.yaml | grep -A 40 'needs:'Repository: NVIDIA/NemoClaw
Length of output: 1654
Add path_instructions entry to .coderabbit.yaml for launchable-smoke-e2e.
The new job was correctly added to notify-on-failure needs list, but no corresponding path_instructions mapping exists in .coderabbit.yaml. Per coding guidelines, every new E2E job requires a path mapping for the source files it covers.
Add an entry that maps test/e2e/test-launchable-smoke.sh to the job name launchable-smoke-e2e in the path_instructions block, following the pattern used by other E2E jobs. Note: the cross-validation test in validate-e2e-coverage.test.ts only warns about uncovered jobs (does not fail), so this mapping must be added manually.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In @.github/workflows/nightly-e2e.yaml around lines 1288 - 1334, Add a new
path_instructions mapping in .coderabbit.yaml associating the test file
test/e2e/test-launchable-smoke.sh with the job name launchable-smoke-e2e; locate
the existing path_instructions block and add an entry following the same pattern
as other E2E jobs (map the file path to the job identifier
"launchable-smoke-e2e") so the notify-on-failure coverage includes this new job.
The redirect after sudo is intentional — we want the log file owned by the runner user, not root. Add disable directive to silence the warning.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
test/e2e/test-launchable-smoke.sh (1)
157-164:⚠️ Potential issue | 🟠 MajorFail fast in clone pre-seed before reporting success.
Line 164 can report PASS even when setup steps fail (Line 162 is explicitly tolerated with
|| true, and other git commands are unchecked). This hides the real install root cause.Suggested fix
-git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR" +if ! git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"; then + fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR" + exit 1 +fi @@ -git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true -git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR" +if ! git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null; then + fail "Failed to create local main branch in pre-seeded clone" + exit 1 +fi +if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then + fail "Failed to repoint origin for pre-seeded clone" + exit 1 +fi pass "Pre-cleanup complete (clone dir pre-seeded)"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/test-launchable-smoke.sh` around lines 157 - 164, The script currently tolerates failures during the clone pre-seed (notably the checkout with `git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true` and unchecked git commands) but still emits pass via `pass "Pre-cleanup complete (clone dir pre-seeded)"`; change this to fail-fast by removing the `|| true` from the checkout, checking the exit status of the `git clone`, `git -C ... checkout -B main HEAD`, and `git -C ... remote set-url origin` commands (or enable `set -e` at the top of the script), and if any of these fail call your existing failure handler instead of `pass` so the script does not report success when pre-seed steps failed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 157-164: The script currently tolerates failures during the clone
pre-seed (notably the checkout with `git -C "$NEMOCLAW_CLONE_DIR" checkout -B
main HEAD 2>/dev/null || true` and unchecked git commands) but still emits pass
via `pass "Pre-cleanup complete (clone dir pre-seeded)"`; change this to
fail-fast by removing the `|| true` from the checkout, checking the exit status
of the `git clone`, `git -C ... checkout -B main HEAD`, and `git -C ... remote
set-url origin` commands (or enable `set -e` at the top of the script), and if
any of these fail call your existing failure handler instead of `pass` so the
script does not report success when pre-seed steps failed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 19d201f5-69ec-4d1e-a34f-d5d3d5990e11
📒 Files selected for processing (1)
test/e2e/test-launchable-smoke.sh
SC1125: shellcheck doesn't parse inline comments after the directive. Move the explanation to a separate comment line above.
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
test/e2e/test-launchable-smoke.sh (1)
157-164:⚠️ Potential issue | 🟠 MajorFail pre-seed setup immediately on git errors.
Line 162 masks failures (
|| true), and Line 164 reports PASS even if pre-seed operations failed. This can silently skip testing the intended checkout.Suggested fix
-git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR" +if ! git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"; then + fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR" + exit 1 +fi # Ensure a "main" branch exists at the current commit for the script's # `git fetch origin main && git checkout main` to succeed. Point origin # at the clone itself so fetch resolves locally (the CI checkout may be # in detached HEAD and lack a "main" branch). -git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true -git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR" +if ! git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null; then + fail "Failed to create/point main branch in pre-seeded clone" + exit 1 +fi +if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then + fail "Failed to repoint origin for pre-seeded clone" + exit 1 +fi pass "Pre-cleanup complete (clone dir pre-seeded)"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@test/e2e/test-launchable-smoke.sh` around lines 157 - 164, The pre-seed git commands currently swallow failures (the `git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true` line) and still call `pass "Pre-cleanup complete (clone dir pre-seeded)"`, so update the script to fail fast: remove the `|| true` from the `git checkout -B main HEAD` call and ensure the subsequent `git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"` is checked for non-zero exit (or enable `set -e` at the top), and only invoke the `pass` helper after both git commands succeed; if either command fails, exit non-zero or call the existing failure helper instead of printing pass.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 149-150: The cleanup currently runs sudo rm -rf
"$NEMOCLAW_CLONE_DIR" with no validation; add a guard that validates
NEMOCLAW_CLONE_DIR before any rm: ensure it is set and non-empty, not "/" or "~"
or "$HOME", and (preferably) resides under an expected safe parent (e.g., a tmp
or workspace prefix) or matches a strict pattern; if the check fails, log an
error and skip deletion (exit non-zero or continue safely). Update both
occurrences that call sudo rm -rf "$NEMOCLAW_CLONE_DIR" (the instance shown and
the one at lines referenced 559-562) to perform this validation using the same
helper check/conditional before calling sudo rm -rf.
- Around line 1-3: The variable TIMEOUT_CMD is assigned but never used,
triggering ShellCheck SC2034; either delete the unused assignment or make it
actually used/exported. To fix, locate the TIMEOUT_CMD variable assignment and
either remove that line if no timeout behavior is needed, or export/use it where
timeouts are applied (e.g., replace direct calls to commands with "$TIMEOUT_CMD
<command>" or add export TIMEOUT_CMD so subshells can access it), ensuring the
symbol TIMEOUT_CMD is referenced after its definition.
---
Duplicate comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 157-164: The pre-seed git commands currently swallow failures (the
`git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true` line)
and still call `pass "Pre-cleanup complete (clone dir pre-seeded)"`, so update
the script to fail fast: remove the `|| true` from the `git checkout -B main
HEAD` call and ensure the subsequent `git -C "$NEMOCLAW_CLONE_DIR" remote
set-url origin "$NEMOCLAW_CLONE_DIR"` is checked for non-zero exit (or enable
`set -e` at the top), and only invoke the `pass` helper after both git commands
succeed; if either command fails, exit non-zero or call the existing failure
helper instead of printing pass.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: af0d67a5-0328-431d-802b-41f835399cf2
📒 Files selected for processing (1)
test/e2e/test-launchable-smoke.sh
| #!/bin/bash | ||
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| # SPDX-License-Identifier: Apache-2.0 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# 1) Verify git mode (expect 100755 for executable shell script)
git ls-files --stage -- test/e2e/test-launchable-smoke.sh
# 2) Verify ShellCheck compliance (if installed)
if command -v shellcheck >/dev/null 2>&1; then
shellcheck test/e2e/test-launchable-smoke.sh
fi
# 3) Verify shfmt formatting (if installed)
if command -v shfmt >/dev/null 2>&1; then
shfmt -d test/e2e/test-launchable-smoke.sh
fiRepository: NVIDIA/NemoClaw
Length of output: 497
Fix ShellCheck warning on line 448.
Executable bit and SPDX headers are correct. However, ShellCheck reports SC2034 on line 448: TIMEOUT_CMD is assigned but appears unused. Either remove the assignment or export/use the variable as intended.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/e2e/test-launchable-smoke.sh` around lines 1 - 3, The variable
TIMEOUT_CMD is assigned but never used, triggering ShellCheck SC2034; either
delete the unused assignment or make it actually used/exported. To fix, locate
the TIMEOUT_CMD variable assignment and either remove that line if no timeout
behavior is needed, or export/use it where timeouts are applied (e.g., replace
direct calls to commands with "$TIMEOUT_CMD <command>" or add export TIMEOUT_CMD
so subshells can access it), ensuring the symbol TIMEOUT_CMD is referenced after
its definition.
| sudo rm -rf "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf "$NEMOCLAW_CLONE_DIR" || true | ||
|
|
There was a problem hiding this comment.
Guard sudo rm -rf against unsafe clone-dir values.
NEMOCLAW_CLONE_DIR is env-configurable, and both cleanup sites recursively delete it with sudo. Add a hard safety check before deletion to prevent destructive paths (/, empty, $HOME, etc.).
Suggested fix
NEMOCLAW_CLONE_DIR="${NEMOCLAW_CLONE_DIR:-${HOME}/NemoClaw-launchable}"
export NEMOCLAW_CLONE_DIR
+
+validate_clone_dir() {
+ case "${NEMOCLAW_CLONE_DIR}" in
+ ""|"/"|". "|"${HOME}"|"/root"|"/home")
+ fail "Refusing unsafe NEMOCLAW_CLONE_DIR='${NEMOCLAW_CLONE_DIR}'"
+ exit 1
+ ;;
+ esac
+}
+validate_clone_dir
@@
-sudo rm -rf "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf "$NEMOCLAW_CLONE_DIR" || true
+sudo rm -rf -- "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf -- "$NEMOCLAW_CLONE_DIR" || true
@@
-sudo rm -rf "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf "$NEMOCLAW_CLONE_DIR" || true
+sudo rm -rf -- "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf -- "$NEMOCLAW_CLONE_DIR" || trueAlso applies to: 559-562
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/e2e/test-launchable-smoke.sh` around lines 149 - 150, The cleanup
currently runs sudo rm -rf "$NEMOCLAW_CLONE_DIR" with no validation; add a guard
that validates NEMOCLAW_CLONE_DIR before any rm: ensure it is set and non-empty,
not "/" or "~" or "$HOME", and (preferably) resides under an expected safe
parent (e.g., a tmp or workspace prefix) or matches a strict pattern; if the
check fails, log an error and skip deletion (exit non-zero or continue safely).
Update both occurrences that call sudo rm -rf "$NEMOCLAW_CLONE_DIR" (the
instance shown and the one at lines referenced 559-562) to perform this
validation using the same helper check/conditional before calling sudo rm -rf.
## Summary `scripts/brev-launchable-ci-cpu.sh` is the community install path for Brev users — it bootstraps a VM with Docker, Node.js, OpenShell, and NemoClaw. **That script already exists in the repo but has zero CI coverage.** This PR adds a nightly E2E smoke test that validates the script works end-to-end. This is the long-living safety net for the community install flow. If any regression breaks the launchable script (e.g., the Apr 20–25 Brev outage from NVIDIA#2472/NVIDIA#2482, or the container reachability fallback from NVIDIA#2425), this test catches it before community users are affected. ## Related Issue Closes NVIDIA#2599 Related: NVIDIA#2425 (the `isProxyHealthy()` fallback in PR NVIDIA#2453 — if that regresses, onboard will abort on Brev and this smoke test catches it) ## Changes ### New: `test/e2e/test-launchable-smoke.sh` | Phase | What it validates | |-------|-------------------| | 0 | Pre-cleanup + pre-seed clone directory from checkout | | 1 | Prerequisites (Docker, NVIDIA_API_KEY, network, env vars) | | 2 | Run `brev-launchable-ci-cpu.sh` — the existing community bootstrap script | | 3 | Verify artifacts (nemoclaw, openshell, Node.js, Docker, sentinel file, built outputs) | | 4 | `nemoclaw onboard --non-interactive` with cloud provider | | 5 | Sandbox health (list, status, inference config, gateway) | | 6 | Live inference (direct API, routing via inference.local, openclaw agent 6×7=42) | | 7 | Destroy + cleanup | Key design decisions: - **No BREV_API_TOKEN needed** — the launchable script is a generic Ubuntu bootstrap with zero Brev dependencies, so it runs on standard GitHub-hosted `ubuntu-latest` runners - **Tests current code, not main** — pre-seeds the clone directory from the CI checkout so regressions are caught before reaching community users - **Follows existing E2E conventions** — pass/fail/section helpers, e2e-timeout.sh self-wrap, sandbox-teardown.sh EXIT trap, parse_chat_content() for reasoning models ### Modified: `.github/workflows/nightly-e2e.yaml` - Added `launchable-smoke-e2e` job: `ubuntu-latest`, 30min timeout, `NVIDIA_API_KEY` secret - Uploads install/onboard/test logs as artifacts on failure - Added to `notify-on-failure` needs list ## Validation Triggered via fork dispatch (`jyaunches/NemoClaw` → `sparky-dispatch` → `launchable-smoke`): - **Run:** https://github.com/jyaunches/NemoClaw/actions/runs/25075715342 - **Result:** ✅ 24 passed, 0 failed, 1 skipped (Node.js version — GH runner pre-installs Node 20) - **Runtime:** ~12 minutes ## Type of Change - [x] Code change (feature, bug fix, or refactor) ## Checklist - [x] Follows project coding conventions - [x] Tests pass locally or in CI - [x] No secrets/credentials committed <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an end-to-end smoke test and CI job that validates the community launchable CPU install path (install, onboarding, runtime readiness, and a simple inference check). CI now uploads install/onboard/test logs on failures. * **Chores** * Renamed the branch-validation workflow and corresponding test-suite identifiers for clarity. * Updated E2E test documentation and project configuration names to match the new labeling. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Summary
scripts/brev-launchable-ci-cpu.shis the community install path for Brev users — it bootstraps a VM with Docker, Node.js, OpenShell, and NemoClaw. That script already exists in the repo but has zero CI coverage. This PR adds a nightly E2E smoke test that validates the script works end-to-end.This is the long-living safety net for the community install flow. If any regression breaks the launchable script (e.g., the Apr 20–25 Brev outage from #2472/#2482, or the container reachability fallback from #2425), this test catches it before community users are affected.
Related Issue
Closes #2599
Related: #2425 (the
isProxyHealthy()fallback in PR #2453 — if that regresses, onboard will abort on Brev and this smoke test catches it)Changes
New:
test/e2e/test-launchable-smoke.shbrev-launchable-ci-cpu.sh— the existing community bootstrap scriptnemoclaw onboard --non-interactivewith cloud providerKey design decisions:
ubuntu-latestrunnersModified:
.github/workflows/nightly-e2e.yamllaunchable-smoke-e2ejob:ubuntu-latest, 30min timeout,NVIDIA_API_KEYsecretnotify-on-failureneeds listValidation
Triggered via fork dispatch (
jyaunches/NemoClaw→sparky-dispatch→launchable-smoke):Type of Change
Checklist
Summary by CodeRabbit
New Features
Chores