Skip to content

test(e2e): add Brev launchable install-flow smoke test#2677

Merged
jyaunches merged 5 commits into
mainfrom
issue-2599-brev-launchable-smoke-test
Apr 29, 2026
Merged

test(e2e): add Brev launchable install-flow smoke test#2677
jyaunches merged 5 commits into
mainfrom
issue-2599-brev-launchable-smoke-test

Conversation

@jyaunches

@jyaunches jyaunches commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Summary

scripts/brev-launchable-ci-cpu.sh is the community install path for Brev users — it bootstraps a VM with Docker, Node.js, OpenShell, and NemoClaw. That script already exists in the repo but has zero CI coverage. This PR adds a nightly E2E smoke test that validates the script works end-to-end.

This is the long-living safety net for the community install flow. If any regression breaks the launchable script (e.g., the Apr 20–25 Brev outage from #2472/#2482, or the container reachability fallback from #2425), this test catches it before community users are affected.

Related Issue

Closes #2599
Related: #2425 (the isProxyHealthy() fallback in PR #2453 — if that regresses, onboard will abort on Brev and this smoke test catches it)

Changes

New: test/e2e/test-launchable-smoke.sh

Phase What it validates
0 Pre-cleanup + pre-seed clone directory from checkout
1 Prerequisites (Docker, NVIDIA_API_KEY, network, env vars)
2 Run brev-launchable-ci-cpu.sh — the existing community bootstrap script
3 Verify artifacts (nemoclaw, openshell, Node.js, Docker, sentinel file, built outputs)
4 nemoclaw onboard --non-interactive with cloud provider
5 Sandbox health (list, status, inference config, gateway)
6 Live inference (direct API, routing via inference.local, openclaw agent 6×7=42)
7 Destroy + cleanup

Key design decisions:

  • No BREV_API_TOKEN needed — the launchable script is a generic Ubuntu bootstrap with zero Brev dependencies, so it runs on standard GitHub-hosted ubuntu-latest runners
  • Tests current code, not main — pre-seeds the clone directory from the CI checkout so regressions are caught before reaching community users
  • Follows existing E2E conventions — pass/fail/section helpers, e2e-timeout.sh self-wrap, sandbox-teardown.sh EXIT trap, parse_chat_content() for reasoning models

Modified: .github/workflows/nightly-e2e.yaml

  • Added launchable-smoke-e2e job: ubuntu-latest, 30min timeout, NVIDIA_API_KEY secret
  • Uploads install/onboard/test logs as artifacts on failure
  • Added to notify-on-failure needs list

Validation

Triggered via fork dispatch (jyaunches/NemoClawsparky-dispatchlaunchable-smoke):

Type of Change

  • Code change (feature, bug fix, or refactor)

Checklist

  • Follows project coding conventions
  • Tests pass locally or in CI
  • No secrets/credentials committed

Summary by CodeRabbit

  • New Features

    • Added an end-to-end smoke test and CI job that validates the community launchable CPU install path (install, onboarding, runtime readiness, and a simple inference check). CI now uploads install/onboard/test logs on failures.
  • Chores

    • Renamed the branch-validation workflow and corresponding test-suite identifiers for clarity.
    • Updated E2E test documentation and project configuration names to match the new labeling.

Add test/e2e/test-launchable-smoke.sh — validates the community install
path (scripts/brev-launchable-ci-cpu.sh) end-to-end on ubuntu-latest:

  Phase 2: Run brev-launchable-ci-cpu.sh with NEMOCLAW_REF=current branch
  Phase 3: Verify artifacts (nemoclaw, openshell, Node ≥22, Docker, sentinel)
  Phase 4: nemoclaw onboard --non-interactive (cloud provider)
  Phase 5: Sandbox health (list, status, inference config, gateway)
  Phase 6: Live inference (direct API, routing via inference.local, openclaw agent)
  Phase 7: Destroy + cleanup

Wire launchable-smoke-e2e into .github/workflows/nightly-e2e.yaml:
  - runs-on: ubuntu-latest, timeout: 30min
  - NVIDIA_API_KEY secret, SKIP_DOCKER_PULL=1
  - Upload install/onboard/test logs as artifacts on failure
  - Added to notify-on-failure needs list

Key insight: brev-launchable-ci-cpu.sh has zero Brev dependencies — it is
a generic Ubuntu bootstrap script that runs on GitHub-hosted runners with
no BREV_API_TOKEN needed.

Closes #2599
@coderabbitai

coderabbitai Bot commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a nightly GitHub Actions job and a new end-to-end Bash smoke test to validate the Brev Launchable community install flow; the job runs on ubuntu-latest, executes test/e2e/test-launchable-smoke.sh, uploads logs on failures, and integrates into nightly reporting and PR notification aggregations.

Changes

Cohort / File(s) Summary
Nightly workflow
.github/workflows/nightly-e2e.yaml
Added launchable-smoke-e2e job (runs on ubuntu-latest), sets Launchable-specific env vars, runs test/e2e/test-launchable-smoke.sh, uploads install/onboard/test artifacts on failure, and added the job to inputs.jobs, notify-on-failure, and report-to-pr aggregations.
E2E smoke test script
test/e2e/test-launchable-smoke.sh
New long-running Bash end-to-end smoke test for the Launchable install path: pre-clean, pre-seed clone, prerequisite checks (Docker, NVIDIA API key, network, env flags), run installer under sudo with streamed logs, runtime readiness checks, non-interactive onboarding, inference/routing checks (direct NVIDIA PONG, HTTPS via sandbox, openclaw agent assertion), optional cleanup, and aggregated result reporting.
Branch validation workflow rename
.github/workflows/e2e-branch-validation.yaml
Renamed workflow and job identifiers from e2e-breve2e-branch-validation; updated concurrency group, job id, Vitest project selection, and artifact naming to match new label.
Vitest config, docs, and test comments
vitest.config.ts, AGENTS.md, test/e2e/brev-e2e.test.ts
Replaced e2e-brev with e2e-branch-validation across Vitest config, documentation, and test comments; adjusted descriptions to reflect branch-validation focus. No runtime logic changes.

Sequence Diagram(s)

sequenceDiagram
    participant Runner as CI Runner
    participant Installer as launch-plugin.sh / test script
    participant Docker as Docker daemon
    participant Sandbox as Brev Sandbox / Gateway
    participant OpenShell as OpenShell / CLI
    participant NVIDIA as integrate.api.nvidia.com

    Runner->>Installer: set envs & execute `test-launchable-smoke.sh`
    Installer->>Docker: ensure Docker running, run installer, start gateway container
    Docker-->>Installer: container started / health OK
    Installer->>Sandbox: run `nemoclaw onboard --non-interactive`
    Sandbox->>OpenShell: expose CLI and inference endpoint
    OpenShell->>NVIDIA: call NVIDIA endpoint (expect "PONG")
    Runner->>Sandbox: SSH/curl https://inference.local (retries)
    Runner->>Sandbox: run `openclaw agent` and assert result == 42
    Runner->>Installer: destroy sandbox/gateway (unless KEEP_SANDBOX)
    Installer-->>Runner: upload logs on failure
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I hopped into night to run the test and see,
The installer danced and containers woke for me,
Endpoints ponged and agents sent a clue,
Forty-two returned — a tiny victory true,
Logs snugly stored, the nightly watch anew.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The PR includes additional scope beyond the primary objective: renaming the e2e-brev workflow/job to e2e-branch-validation in multiple files (vitest.config.ts, .github/workflows/e2e-branch-validation.yaml, AGENTS.md, brev-e2e.test.ts), which is unrelated to adding the Launchable smoke test. Extract the e2e-brev rename refactoring into a separate PR to maintain focused scope; this PR should focus solely on implementing the Launchable smoke test as described in #2599.
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the primary change: adding a Brev launchable install-flow smoke test for E2E validation, which matches the main objective of the PR.
Linked Issues check ✅ Passed The PR successfully implements all key coding requirements from #2599: creates test/e2e/test-launchable-smoke.sh to validate the Launchable install flow, adds launchable-smoke-e2e job to nightly workflow, verifies NemoClaw/sandbox health, performs live inference tests, and validates the install path.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-2599-brev-launchable-smoke-test

Review rate limit: 7/10 reviews remaining, refill in 13 minutes and 54 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
test/e2e/test-launchable-smoke.sh (1)

444-446: Remove dead TIMEOUT_CMD assignments (SC2034).

TIMEOUT_CMD is assigned on Line 444-446 but never used. This is dead code and keeps ShellCheck noisy.

Suggested fix
-TIMEOUT_CMD=""
-command -v timeout >/dev/null 2>&1 && TIMEOUT_CMD="timeout"
-command -v gtimeout >/dev/null 2>&1 && TIMEOUT_CMD="gtimeout"

As per coding guidelines, **/*.sh files must include ShellCheck compliance.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/test-launchable-smoke.sh` around lines 444 - 446, The variable
TIMEOUT_CMD is assigned via the command -v checks but never used, causing dead
code and ShellCheck SC2034; remove the three lines that declare/assign
TIMEOUT_CMD (the variable and the two command -v checks) from the script, or if
a timeout wrapper was intended, replace the dead assignments by actually using
TIMEOUT_CMD where timeouts are invoked (e.g., use "$TIMEOUT_CMD" before
commands) or export a single explicit timeout command; specifically locate the
occurrences of the symbol TIMEOUT_CMD and delete the assignment block if unused,
or wire it into timeout invocations to eliminate the dead-variable warning.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Line 223: SC2024 is triggered by redirecting output outside sudo for the
backgrounded invocation of brev-launchable-ci-cpu.sh; fix by running the
redirection inside the elevated shell so the file is created/owned under the
sudo context, e.g. replace the current command with a sudo -E bash -c "exec
\"$REPO/scripts/brev-launchable-ci-cpu.sh\" >\"$INSTALL_LOG\" 2>&1 &" form
(ensuring INSTALL_LOG is quoted), or if the original behavior of creating a
runner-owned log file is intentional, add a targeted ShellCheck suppression
comment (# shellcheck disable=SC2024) immediately above that line with a short
rationale referencing SC2024 and INSTALL_LOG.
- Around line 157-164: The script currently always calls pass "Pre-cleanup
complete (clone dir pre-seeded)" even if the preceding setup steps fail; make
the clone/pre-seed commands fail fast and only call pass on success: remove the
tolerant "|| true" on the git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD
command (or replace it with explicit error handling), ensure the git clone and
git -C ... remote set-url origin commands are checked for non-zero exit codes
(or enable a shell-wide "set -e" at the top of the script), and if any of those
commands fail invoke the script's existing failure handler (or exit non-zero)
instead of proceeding to call pass "Pre-cleanup complete (clone dir
pre-seeded)"; reference the NEMOCLAW_CLONE_DIR variable and the pass invocation
to locate the lines to change.

---

Nitpick comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 444-446: The variable TIMEOUT_CMD is assigned via the command -v
checks but never used, causing dead code and ShellCheck SC2034; remove the three
lines that declare/assign TIMEOUT_CMD (the variable and the two command -v
checks) from the script, or if a timeout wrapper was intended, replace the dead
assignments by actually using TIMEOUT_CMD where timeouts are invoked (e.g., use
"$TIMEOUT_CMD" before commands) or export a single explicit timeout command;
specifically locate the occurrences of the symbol TIMEOUT_CMD and delete the
assignment block if unused, or wire it into timeout invocations to eliminate the
dead-variable warning.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b3e45d3d-7c18-475b-8ce8-9da6768fab3e

📥 Commits

Reviewing files that changed from the base of the PR and between b4ef3db and 04edd8d.

📒 Files selected for processing (2)
  • .github/workflows/nightly-e2e.yaml
  • test/e2e/test-launchable-smoke.sh

Comment on lines +157 to +164
git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
# Ensure a "main" branch exists at the current commit for the script's
# `git fetch origin main && git checkout main` to succeed. Point origin
# at the clone itself so fetch resolves locally (the CI checkout may be
# in detached HEAD and lack a "main" branch).
git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"
pass "Pre-cleanup complete (clone dir pre-seeded)"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Fail fast on clone pre-seed errors before marking PASS.

Line 157, Line 162, and Line 163 are all critical setup operations, but Line 164 always records success even if one of those commands fails. That can hide root-cause install failures and make the smoke test report misleading.

Suggested fix
 git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
+if [ $? -ne 0 ]; then
+  fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR"
+  exit 1
+fi
 # Ensure a "main" branch exists at the current commit for the script's
 # `git fetch origin main && git checkout main` to succeed. Point origin
 # at the clone itself so fetch resolves locally (the CI checkout may be
 # in detached HEAD and lack a "main" branch).
 git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
-git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"
+if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then
+  fail "Failed to repoint origin for pre-seeded clone"
+  exit 1
+fi
 pass "Pre-cleanup complete (clone dir pre-seeded)"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
# Ensure a "main" branch exists at the current commit for the script's
# `git fetch origin main && git checkout main` to succeed. Point origin
# at the clone itself so fetch resolves locally (the CI checkout may be
# in detached HEAD and lack a "main" branch).
git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"
pass "Pre-cleanup complete (clone dir pre-seeded)"
git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
if [ $? -ne 0 ]; then
fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR"
exit 1
fi
# Ensure a "main" branch exists at the current commit for the script's
# `git fetch origin main && git checkout main` to succeed. Point origin
# at the clone itself so fetch resolves locally (the CI checkout may be
# in detached HEAD and lack a "main" branch).
git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then
fail "Failed to repoint origin for pre-seeded clone"
exit 1
fi
pass "Pre-cleanup complete (clone dir pre-seeded)"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/test-launchable-smoke.sh` around lines 157 - 164, The script
currently always calls pass "Pre-cleanup complete (clone dir pre-seeded)" even
if the preceding setup steps fail; make the clone/pre-seed commands fail fast
and only call pass on success: remove the tolerant "|| true" on the git -C
"$NEMOCLAW_CLONE_DIR" checkout -B main HEAD command (or replace it with explicit
error handling), ensure the git clone and git -C ... remote set-url origin
commands are checked for non-zero exit codes (or enable a shell-wide "set -e" at
the top of the script), and if any of those commands fail invoke the script's
existing failure handler (or exit non-zero) instead of proceeding to call pass
"Pre-cleanup complete (clone dir pre-seeded)"; reference the NEMOCLAW_CLONE_DIR
variable and the pass invocation to locate the lines to change.

Comment thread test/e2e/test-launchable-smoke.sh
Rename the existing Brev E2E workflow to clearly communicate its purpose:
validating a git branch by installing from source on a clean machine.

This distinguishes it from the new e2e-launchable-smoke job (added in the
prior commit) which validates the community install path that real Brev
Launchable users run.

Changes:
- Rename .github/workflows/e2e-brev.yaml → e2e-branch-validation.yaml
- Update workflow name, job name, concurrency group, artifact names
- Add expanded header comments explaining purpose, how/when to use
- Update vitest project name: e2e-brev → e2e-branch-validation
- Update brev-e2e.test.ts doc comment with clear purpose + cross-ref
- Update AGENTS.md reference

Part of #2599

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.github/workflows/e2e-branch-validation.yaml (1)

120-121: Concurrency fallback currently disables dedupe for non-PR runs.

When inputs.pr_number is empty, using github.run_id makes the group unique every run, so cancel-in-progress never helps for branch-based workflow_call runs.

Suggested patch
-concurrency:
-  group: e2e-branch-validation-${{ inputs.pr_number || github.run_id }}
+concurrency:
+  group: e2e-branch-validation-${{ inputs.pr_number || inputs.branch || github.ref_name || github.run_id }}
   cancel-in-progress: true
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/e2e-branch-validation.yaml around lines 120 - 121, The
concurrency group uses a per-run unique fallback (github.run_id) so
cancel-in-progress never dedupes non-PR workflow_call runs; change the
expression used in the concurrency group from e2e-branch-validation-${{
inputs.pr_number || github.run_id }} to a stable branch/ref fallback such as
e2e-branch-validation-${{ inputs.pr_number || github.ref }} (or github.ref_name)
so branch-based runs share the same group; update the group string accordingly
where "group: e2e-branch-validation-${{ inputs.pr_number || github.run_id }}"
appears and keep "cancel-in-progress: true".
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/e2e-branch-validation.yaml:
- Around line 25-27: Update the header comment that currently references
"e2e-launchable-smoke" to the correct job identifier "launchable-smoke-e2e" so
the note matches this PR's context; search for the string "e2e-launchable-smoke"
in the header comment near the top of
.github/workflows/e2e-branch-validation.yaml and replace it with
"launchable-smoke-e2e" to avoid misrouting maintainers.

---

Nitpick comments:
In @.github/workflows/e2e-branch-validation.yaml:
- Around line 120-121: The concurrency group uses a per-run unique fallback
(github.run_id) so cancel-in-progress never dedupes non-PR workflow_call runs;
change the expression used in the concurrency group from
e2e-branch-validation-${{ inputs.pr_number || github.run_id }} to a stable
branch/ref fallback such as e2e-branch-validation-${{ inputs.pr_number ||
github.ref }} (or github.ref_name) so branch-based runs share the same group;
update the group string accordingly where "group: e2e-branch-validation-${{
inputs.pr_number || github.run_id }}" appears and keep "cancel-in-progress:
true".
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5726259b-4065-46cc-bad7-496829a341d3

📥 Commits

Reviewing files that changed from the base of the PR and between 04edd8d and 16490de.

📒 Files selected for processing (4)
  • .github/workflows/e2e-branch-validation.yaml
  • AGENTS.md
  • test/e2e/brev-e2e.test.ts
  • vitest.config.ts
✅ Files skipped from review due to trivial changes (3)
  • AGENTS.md
  • vitest.config.ts
  • test/e2e/brev-e2e.test.ts

Comment on lines +25 to +27
# NOTE: This does NOT test the community install path (launch-plugin.sh).
# For validating what Brev Launchable users actually get, see the
# e2e-launchable-smoke job in nightly-e2e.yaml.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix stale launchable job identifier in the header note.

The comment references e2e-launchable-smoke, but this PR context uses launchable-smoke-e2e. Keeping this exact avoids misrouting maintainers.

Suggested patch
-# e2e-launchable-smoke job in nightly-e2e.yaml.
+# launchable-smoke-e2e job in nightly-e2e.yaml.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# NOTE: This does NOT test the community install path (launch-plugin.sh).
# For validating what Brev Launchable users actually get, see the
# e2e-launchable-smoke job in nightly-e2e.yaml.
# NOTE: This does NOT test the community install path (launch-plugin.sh).
# For validating what Brev Launchable users actually get, see the
# launchable-smoke-e2e job in nightly-e2e.yaml.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/e2e-branch-validation.yaml around lines 25 - 27, Update
the header comment that currently references "e2e-launchable-smoke" to the
correct job identifier "launchable-smoke-e2e" so the note matches this PR's
context; search for the string "e2e-launchable-smoke" in the header comment near
the top of .github/workflows/e2e-branch-validation.yaml and replace it with
"launchable-smoke-e2e" to avoid misrouting maintainers.

Resolve conflict with credential-migration-e2e comment block from main.
Add selective dispatch guard to launchable-smoke-e2e to match the pattern
from PR #2637. Add launchable-smoke-e2e to the valid jobs input list.
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 25125393577
Branch: issue-2599-brev-launchable-smoke-test
Requested jobs: launchable-smoke-e2e
Summary: 0 passed, 0 failed, 22 skipped

Job Result
cloud-e2e ⏭️ skipped
cloud-inference-e2e ⏭️ skipped
cloud-onboard-e2e ⏭️ skipped
deployment-services-e2e ⏭️ skipped
diagnostics-e2e ⏭️ skipped
docs-validation-e2e ⏭️ skipped
gpu-e2e ⏭️ skipped
hermes-e2e ⏭️ skipped
inference-routing-e2e ⏭️ skipped
messaging-providers-e2e ⏭️ skipped
network-policy-e2e ⏭️ skipped
overlayfs-autofix-e2e ⏭️ skipped
rebuild-hermes-e2e ⏭️ skipped
rebuild-openclaw-e2e ⏭️ skipped
sandbox-operations-e2e ⏭️ skipped
sandbox-survival-e2e ⏭️ skipped
shields-config-e2e ⏭️ skipped
skill-agent-e2e ⏭️ skipped
skip-permissions-e2e ⏭️ skipped
snapshot-commands-e2e ⏭️ skipped
token-rotation-e2e ⏭️ skipped
upgrade-stale-sandbox-e2e ⏭️ skipped

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/nightly-e2e.yaml:
- Around line 1288-1294: The new job launchable-smoke-e2e must be added to the
dependency list for the report-to-pr job so the PR summary waits for it; update
the report-to-pr job's needs array (referencing the job name report-to-pr and
the new launchable-smoke-e2e job) to include "launchable-smoke-e2e" alongside
the existing needs entries so report-to-pr runs only after that job completes.
- Around line 1288-1334: Add a new path_instructions mapping in .coderabbit.yaml
associating the test file test/e2e/test-launchable-smoke.sh with the job name
launchable-smoke-e2e; locate the existing path_instructions block and add an
entry following the same pattern as other E2E jobs (map the file path to the job
identifier "launchable-smoke-e2e") so the notify-on-failure coverage includes
this new job.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b97e7663-db60-4f8c-8115-2016615e1fa4

📥 Commits

Reviewing files that changed from the base of the PR and between 16490de and 1188018.

📒 Files selected for processing (3)
  • .github/workflows/nightly-e2e.yaml
  • AGENTS.md
  • vitest.config.ts
✅ Files skipped from review due to trivial changes (2)
  • AGENTS.md
  • vitest.config.ts

Comment on lines +1288 to +1294
launchable-smoke-e2e:
if: >-
github.repository == 'NVIDIA/NemoClaw' &&
(github.event_name != 'workflow_dispatch' ||
inputs.jobs == '' ||
contains(format(',{0},', inputs.jobs), ',launchable-smoke-e2e,'))
runs-on: ubuntu-latest

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add launchable-smoke-e2e to report-to-pr.needs to avoid incomplete PR results.

Line 1288 introduces a new runnable job, but report-to-pr (Lines 1531-1555) does not depend on it. That can post a PR summary before this job finishes and omit its status.

🔧 Proposed fix
   report-to-pr:
     runs-on: ubuntu-latest
     needs:
       [
         cloud-e2e,
         cloud-onboard-e2e,
         cloud-inference-e2e,
         skill-agent-e2e,
         docs-validation-e2e,
         messaging-providers-e2e,
         token-rotation-e2e,
         sandbox-survival-e2e,
         hermes-e2e,
         skip-permissions-e2e,
         sandbox-operations-e2e,
         inference-routing-e2e,
         network-policy-e2e,
         deployment-services-e2e,
         diagnostics-e2e,
         snapshot-commands-e2e,
         shields-config-e2e,
         rebuild-openclaw-e2e,
         upgrade-stale-sandbox-e2e,
         rebuild-hermes-e2e,
         overlayfs-autofix-e2e,
+        launchable-smoke-e2e,
         gpu-e2e,
       ]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
launchable-smoke-e2e:
if: >-
github.repository == 'NVIDIA/NemoClaw' &&
(github.event_name != 'workflow_dispatch' ||
inputs.jobs == '' ||
contains(format(',{0},', inputs.jobs), ',launchable-smoke-e2e,'))
runs-on: ubuntu-latest
report-to-pr:
runs-on: ubuntu-latest
needs:
[
cloud-e2e,
cloud-onboard-e2e,
cloud-inference-e2e,
skill-agent-e2e,
docs-validation-e2e,
messaging-providers-e2e,
token-rotation-e2e,
sandbox-survival-e2e,
hermes-e2e,
skip-permissions-e2e,
sandbox-operations-e2e,
inference-routing-e2e,
network-policy-e2e,
deployment-services-e2e,
diagnostics-e2e,
snapshot-commands-e2e,
shields-config-e2e,
rebuild-openclaw-e2e,
upgrade-stale-sandbox-e2e,
rebuild-hermes-e2e,
overlayfs-autofix-e2e,
launchable-smoke-e2e,
gpu-e2e,
]
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/nightly-e2e.yaml around lines 1288 - 1294, The new job
launchable-smoke-e2e must be added to the dependency list for the report-to-pr
job so the PR summary waits for it; update the report-to-pr job's needs array
(referencing the job name report-to-pr and the new launchable-smoke-e2e job) to
include "launchable-smoke-e2e" alongside the existing needs entries so
report-to-pr runs only after that job completes.

Comment on lines +1288 to +1334
launchable-smoke-e2e:
if: >-
github.repository == 'NVIDIA/NemoClaw' &&
(github.event_name != 'workflow_dispatch' ||
inputs.jobs == '' ||
contains(format(',{0},', inputs.jobs), ',launchable-smoke-e2e,'))
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Run launchable install-flow smoke test
env:
NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}
NEMOCLAW_NON_INTERACTIVE: "1"
NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE: "1"
NEMOCLAW_SANDBOX_NAME: "e2e-launchable"
NEMOCLAW_RECREATE_SANDBOX: "1"
SKIP_DOCKER_PULL: "1"
GITHUB_TOKEN: ${{ github.token }}
run: bash test/e2e/test-launchable-smoke.sh

- name: Upload install log on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: launchable-smoke-install-log
path: /tmp/nemoclaw-launchable-install.log
if-no-files-found: ignore

- name: Upload onboard log on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: launchable-smoke-onboard-log
path: /tmp/nemoclaw-launchable-onboard.log
if-no-files-found: ignore

- name: Upload test log on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: launchable-smoke-test-log
path: /tmp/nemoclaw-launchable-test.log
if-no-files-found: ignore

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

cfg="$(fd -a '^\.coderabbit\.yaml$' | head -n1 || true)"
if [ -z "$cfg" ]; then
  echo "ERROR: .coderabbit.yaml not found"
  exit 1
fi

echo "== Checking .coderabbit.yaml for launchable smoke coverage =="
rg -n -C3 'path_instructions|launchable-smoke-e2e|test/e2e/test-launchable-smoke.sh|brev-launchable-ci-cpu.sh' "$cfg"

echo "== Checking coverage validator test references =="
testfile="$(fd -a '^validate-e2e-coverage\.test\.ts$' | head -n1 || true)"
if [ -n "$testfile" ]; then
  rg -n -C3 'launchable-smoke-e2e|test-launchable-smoke' "$testfile"
else
  echo "WARN: validate-e2e-coverage.test.ts not found"
fi

Repository: NVIDIA/NemoClaw

Length of output: 319


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find .coderabbit.yaml in root with proper hidden file handling
cfg="$(find . -maxdepth 1 -name '.coderabbit.yaml' | head -n1 || true)"
if [ -z "$cfg" ]; then
  echo "ERROR: .coderabbit.yaml not found in repository root"
  ls -la . | head -20
  exit 1
fi

echo "== .coderabbit.yaml found at: $cfg =="
echo ""
echo "== Searching for launchable-smoke-e2e coverage in .coderabbit.yaml =="
rg 'launchable-smoke-e2e' "$cfg" || echo "WARN: launchable-smoke-e2e not found in .coderabbit.yaml"

echo ""
echo "== Checking full path_instructions section for E2E jobs =="
rg -A 200 'path_instructions:' "$cfg" | head -100

echo ""
echo "== Checking if test script exists =="
test_script="test/e2e/test-launchable-smoke.sh"
if [ -f "$test_script" ]; then
  echo "✓ Test script exists: $test_script"
else
  echo "✗ Test script NOT found: $test_script"
fi

echo ""
echo "== Checking validate-e2e-coverage.test.ts =="
testfile="$(find . -name 'validate-e2e-coverage.test.ts' | head -n1 || true)"
if [ -n "$testfile" ]; then
  echo "Found: $testfile"
  rg 'launchable-smoke' "$testfile" || echo "WARN: launchable-smoke reference not found in test file"
else
  echo "WARN: validate-e2e-coverage.test.ts not found"
fi

Repository: NVIDIA/NemoClaw

Length of output: 5756


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Full validate-e2e-coverage.test.ts content =="
cat test/validate-e2e-coverage.test.ts

echo ""
echo "== Search for other E2E job entries in .coderabbit.yaml =="
rg -n 'e2e|smoke|test/' .coderabbit.yaml | head -50

echo ""
echo "== Check for notify-on-failure in .coderabbit.yaml =="
rg -B 5 -A 20 'notify-on-failure' .coderabbit.yaml || echo "WARN: notify-on-failure not found"

Repository: NVIDIA/NemoClaw

Length of output: 13179


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Checking notify-on-failure in nightly-e2e.yaml =="
rg -B 3 -A 30 'notify-on-failure:' .github/workflows/nightly-e2e.yaml | head -80

Repository: NVIDIA/NemoClaw

Length of output: 1063


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== Check if launchable-smoke-e2e appears in nightly-e2e.yaml =="
rg 'launchable-smoke-e2e' .github/workflows/nightly-e2e.yaml

echo ""
echo "== Full notify-on-failure needs list =="
rg -A 50 'notify-on-failure:' .github/workflows/nightly-e2e.yaml | grep -A 40 'needs:'

Repository: NVIDIA/NemoClaw

Length of output: 1654


Add path_instructions entry to .coderabbit.yaml for launchable-smoke-e2e.

The new job was correctly added to notify-on-failure needs list, but no corresponding path_instructions mapping exists in .coderabbit.yaml. Per coding guidelines, every new E2E job requires a path mapping for the source files it covers.

Add an entry that maps test/e2e/test-launchable-smoke.sh to the job name launchable-smoke-e2e in the path_instructions block, following the pattern used by other E2E jobs. Note: the cross-validation test in validate-e2e-coverage.test.ts only warns about uncovered jobs (does not fail), so this mapping must be added manually.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/nightly-e2e.yaml around lines 1288 - 1334, Add a new
path_instructions mapping in .coderabbit.yaml associating the test file
test/e2e/test-launchable-smoke.sh with the job name launchable-smoke-e2e; locate
the existing path_instructions block and add an entry following the same pattern
as other E2E jobs (map the file path to the job identifier
"launchable-smoke-e2e") so the notify-on-failure coverage includes this new job.

The redirect after sudo is intentional — we want the log file owned by
the runner user, not root. Add disable directive to silence the warning.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
test/e2e/test-launchable-smoke.sh (1)

157-164: ⚠️ Potential issue | 🟠 Major

Fail fast in clone pre-seed before reporting success.

Line 164 can report PASS even when setup steps fail (Line 162 is explicitly tolerated with || true, and other git commands are unchecked). This hides the real install root cause.

Suggested fix
-git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
+if ! git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"; then
+  fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR"
+  exit 1
+fi
@@
-git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
-git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"
+if ! git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null; then
+  fail "Failed to create local main branch in pre-seeded clone"
+  exit 1
+fi
+if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then
+  fail "Failed to repoint origin for pre-seeded clone"
+  exit 1
+fi
 pass "Pre-cleanup complete (clone dir pre-seeded)"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/test-launchable-smoke.sh` around lines 157 - 164, The script
currently tolerates failures during the clone pre-seed (notably the checkout
with `git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true`
and unchecked git commands) but still emits pass via `pass "Pre-cleanup complete
(clone dir pre-seeded)"`; change this to fail-fast by removing the `|| true`
from the checkout, checking the exit status of the `git clone`, `git -C ...
checkout -B main HEAD`, and `git -C ... remote set-url origin` commands (or
enable `set -e` at the top of the script), and if any of these fail call your
existing failure handler instead of `pass` so the script does not report success
when pre-seed steps failed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 157-164: The script currently tolerates failures during the clone
pre-seed (notably the checkout with `git -C "$NEMOCLAW_CLONE_DIR" checkout -B
main HEAD 2>/dev/null || true` and unchecked git commands) but still emits pass
via `pass "Pre-cleanup complete (clone dir pre-seeded)"`; change this to
fail-fast by removing the `|| true` from the checkout, checking the exit status
of the `git clone`, `git -C ... checkout -B main HEAD`, and `git -C ... remote
set-url origin` commands (or enable `set -e` at the top of the script), and if
any of these fail call your existing failure handler instead of `pass` so the
script does not report success when pre-seed steps failed.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 19d201f5-69ec-4d1e-a34f-d5d3d5990e11

📥 Commits

Reviewing files that changed from the base of the PR and between 1188018 and ceff955.

📒 Files selected for processing (1)
  • test/e2e/test-launchable-smoke.sh

SC1125: shellcheck doesn't parse inline comments after the directive.
Move the explanation to a separate comment line above.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
test/e2e/test-launchable-smoke.sh (1)

157-164: ⚠️ Potential issue | 🟠 Major

Fail pre-seed setup immediately on git errors.

Line 162 masks failures (|| true), and Line 164 reports PASS even if pre-seed operations failed. This can silently skip testing the intended checkout.

Suggested fix
-git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"
+if ! git clone --local --no-hardlinks "$REPO" "$NEMOCLAW_CLONE_DIR"; then
+  fail "Failed to pre-seed clone dir: $NEMOCLAW_CLONE_DIR"
+  exit 1
+fi
 # Ensure a "main" branch exists at the current commit for the script's
 # `git fetch origin main && git checkout main` to succeed. Point origin
 # at the clone itself so fetch resolves locally (the CI checkout may be
 # in detached HEAD and lack a "main" branch).
-git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true
-git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"
+if ! git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null; then
+  fail "Failed to create/point main branch in pre-seeded clone"
+  exit 1
+fi
+if ! git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"; then
+  fail "Failed to repoint origin for pre-seeded clone"
+  exit 1
+fi
 pass "Pre-cleanup complete (clone dir pre-seeded)"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/test-launchable-smoke.sh` around lines 157 - 164, The pre-seed git
commands currently swallow failures (the `git -C "$NEMOCLAW_CLONE_DIR" checkout
-B main HEAD 2>/dev/null || true` line) and still call `pass "Pre-cleanup
complete (clone dir pre-seeded)"`, so update the script to fail fast: remove the
`|| true` from the `git checkout -B main HEAD` call and ensure the subsequent
`git -C "$NEMOCLAW_CLONE_DIR" remote set-url origin "$NEMOCLAW_CLONE_DIR"` is
checked for non-zero exit (or enable `set -e` at the top), and only invoke the
`pass` helper after both git commands succeed; if either command fails, exit
non-zero or call the existing failure helper instead of printing pass.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 149-150: The cleanup currently runs sudo rm -rf
"$NEMOCLAW_CLONE_DIR" with no validation; add a guard that validates
NEMOCLAW_CLONE_DIR before any rm: ensure it is set and non-empty, not "/" or "~"
or "$HOME", and (preferably) resides under an expected safe parent (e.g., a tmp
or workspace prefix) or matches a strict pattern; if the check fails, log an
error and skip deletion (exit non-zero or continue safely). Update both
occurrences that call sudo rm -rf "$NEMOCLAW_CLONE_DIR" (the instance shown and
the one at lines referenced 559-562) to perform this validation using the same
helper check/conditional before calling sudo rm -rf.
- Around line 1-3: The variable TIMEOUT_CMD is assigned but never used,
triggering ShellCheck SC2034; either delete the unused assignment or make it
actually used/exported. To fix, locate the TIMEOUT_CMD variable assignment and
either remove that line if no timeout behavior is needed, or export/use it where
timeouts are applied (e.g., replace direct calls to commands with "$TIMEOUT_CMD
<command>" or add export TIMEOUT_CMD so subshells can access it), ensuring the
symbol TIMEOUT_CMD is referenced after its definition.

---

Duplicate comments:
In `@test/e2e/test-launchable-smoke.sh`:
- Around line 157-164: The pre-seed git commands currently swallow failures (the
`git -C "$NEMOCLAW_CLONE_DIR" checkout -B main HEAD 2>/dev/null || true` line)
and still call `pass "Pre-cleanup complete (clone dir pre-seeded)"`, so update
the script to fail fast: remove the `|| true` from the `git checkout -B main
HEAD` call and ensure the subsequent `git -C "$NEMOCLAW_CLONE_DIR" remote
set-url origin "$NEMOCLAW_CLONE_DIR"` is checked for non-zero exit (or enable
`set -e` at the top), and only invoke the `pass` helper after both git commands
succeed; if either command fails, exit non-zero or call the existing failure
helper instead of printing pass.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: af0d67a5-0328-431d-802b-41f835399cf2

📥 Commits

Reviewing files that changed from the base of the PR and between ceff955 and becf6b3.

📒 Files selected for processing (1)
  • test/e2e/test-launchable-smoke.sh

Comment on lines +1 to +3
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Verify git mode (expect 100755 for executable shell script)
git ls-files --stage -- test/e2e/test-launchable-smoke.sh

# 2) Verify ShellCheck compliance (if installed)
if command -v shellcheck >/dev/null 2>&1; then
  shellcheck test/e2e/test-launchable-smoke.sh
fi

# 3) Verify shfmt formatting (if installed)
if command -v shfmt >/dev/null 2>&1; then
  shfmt -d test/e2e/test-launchable-smoke.sh
fi

Repository: NVIDIA/NemoClaw

Length of output: 497


Fix ShellCheck warning on line 448.

Executable bit and SPDX headers are correct. However, ShellCheck reports SC2034 on line 448: TIMEOUT_CMD is assigned but appears unused. Either remove the assignment or export/use the variable as intended.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/test-launchable-smoke.sh` around lines 1 - 3, The variable
TIMEOUT_CMD is assigned but never used, triggering ShellCheck SC2034; either
delete the unused assignment or make it actually used/exported. To fix, locate
the TIMEOUT_CMD variable assignment and either remove that line if no timeout
behavior is needed, or export/use it where timeouts are applied (e.g., replace
direct calls to commands with "$TIMEOUT_CMD <command>" or add export TIMEOUT_CMD
so subshells can access it), ensuring the symbol TIMEOUT_CMD is referenced after
its definition.

Comment on lines +149 to +150
sudo rm -rf "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf "$NEMOCLAW_CLONE_DIR" || true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Guard sudo rm -rf against unsafe clone-dir values.

NEMOCLAW_CLONE_DIR is env-configurable, and both cleanup sites recursively delete it with sudo. Add a hard safety check before deletion to prevent destructive paths (/, empty, $HOME, etc.).

Suggested fix
 NEMOCLAW_CLONE_DIR="${NEMOCLAW_CLONE_DIR:-${HOME}/NemoClaw-launchable}"
 export NEMOCLAW_CLONE_DIR
+
+validate_clone_dir() {
+  case "${NEMOCLAW_CLONE_DIR}" in
+    ""|"/"|". "|"${HOME}"|"/root"|"/home")
+      fail "Refusing unsafe NEMOCLAW_CLONE_DIR='${NEMOCLAW_CLONE_DIR}'"
+      exit 1
+      ;;
+  esac
+}
+validate_clone_dir
@@
-sudo rm -rf "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf "$NEMOCLAW_CLONE_DIR" || true
+sudo rm -rf -- "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf -- "$NEMOCLAW_CLONE_DIR" || true
@@
-sudo rm -rf "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf "$NEMOCLAW_CLONE_DIR" || true
+sudo rm -rf -- "$NEMOCLAW_CLONE_DIR" 2>/dev/null || rm -rf -- "$NEMOCLAW_CLONE_DIR" || true

Also applies to: 559-562

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e/test-launchable-smoke.sh` around lines 149 - 150, The cleanup
currently runs sudo rm -rf "$NEMOCLAW_CLONE_DIR" with no validation; add a guard
that validates NEMOCLAW_CLONE_DIR before any rm: ensure it is set and non-empty,
not "/" or "~" or "$HOME", and (preferably) resides under an expected safe
parent (e.g., a tmp or workspace prefix) or matches a strict pattern; if the
check fails, log an error and skip deletion (exit non-zero or continue safely).
Update both occurrences that call sudo rm -rf "$NEMOCLAW_CLONE_DIR" (the
instance shown and the one at lines referenced 559-562) to perform this
validation using the same helper check/conditional before calling sudo rm -rf.

@jyaunches jyaunches merged commit 502d725 into main Apr 29, 2026
19 checks passed
DemianHeyGen pushed a commit to DemianHeyGen/NemoClaw that referenced this pull request Apr 30, 2026
## Summary

`scripts/brev-launchable-ci-cpu.sh` is the community install path for
Brev users — it bootstraps a VM with Docker, Node.js, OpenShell, and
NemoClaw. **That script already exists in the repo but has zero CI
coverage.** This PR adds a nightly E2E smoke test that validates the
script works end-to-end.

This is the long-living safety net for the community install flow. If
any regression breaks the launchable script (e.g., the Apr 20–25 Brev
outage from NVIDIA#2472/NVIDIA#2482, or the container reachability fallback from
NVIDIA#2425), this test catches it before community users are affected.

## Related Issue

Closes NVIDIA#2599
Related: NVIDIA#2425 (the `isProxyHealthy()` fallback in PR NVIDIA#2453 — if that
regresses, onboard will abort on Brev and this smoke test catches it)

## Changes

### New: `test/e2e/test-launchable-smoke.sh`

| Phase | What it validates |
|-------|-------------------|
| 0 | Pre-cleanup + pre-seed clone directory from checkout |
| 1 | Prerequisites (Docker, NVIDIA_API_KEY, network, env vars) |
| 2 | Run `brev-launchable-ci-cpu.sh` — the existing community bootstrap
script |
| 3 | Verify artifacts (nemoclaw, openshell, Node.js, Docker, sentinel
file, built outputs) |
| 4 | `nemoclaw onboard --non-interactive` with cloud provider |
| 5 | Sandbox health (list, status, inference config, gateway) |
| 6 | Live inference (direct API, routing via inference.local, openclaw
agent 6×7=42) |
| 7 | Destroy + cleanup |

Key design decisions:
- **No BREV_API_TOKEN needed** — the launchable script is a generic
Ubuntu bootstrap with zero Brev dependencies, so it runs on standard
GitHub-hosted `ubuntu-latest` runners
- **Tests current code, not main** — pre-seeds the clone directory from
the CI checkout so regressions are caught before reaching community
users
- **Follows existing E2E conventions** — pass/fail/section helpers,
e2e-timeout.sh self-wrap, sandbox-teardown.sh EXIT trap,
parse_chat_content() for reasoning models

### Modified: `.github/workflows/nightly-e2e.yaml`

- Added `launchable-smoke-e2e` job: `ubuntu-latest`, 30min timeout,
`NVIDIA_API_KEY` secret
- Uploads install/onboard/test logs as artifacts on failure
- Added to `notify-on-failure` needs list

## Validation

Triggered via fork dispatch (`jyaunches/NemoClaw` → `sparky-dispatch` →
`launchable-smoke`):
- **Run:**
https://github.com/jyaunches/NemoClaw/actions/runs/25075715342
- **Result:** ✅ 24 passed, 0 failed, 1 skipped (Node.js version — GH
runner pre-installs Node 20)
- **Runtime:** ~12 minutes

## Type of Change
- [x] Code change (feature, bug fix, or refactor)

## Checklist
- [x] Follows project coding conventions
- [x] Tests pass locally or in CI
- [x] No secrets/credentials committed


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added an end-to-end smoke test and CI job that validates the community
launchable CPU install path (install, onboarding, runtime readiness, and
a simple inference check). CI now uploads install/onboard/test logs on
failures.

* **Chores**
* Renamed the branch-validation workflow and corresponding test-suite
identifiers for clarity.
* Updated E2E test documentation and project configuration names to
match the new labeling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@wscurran wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026
@jyaunches jyaunches deleted the issue-2599-brev-launchable-smoke-test branch June 12, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci(e2e): add Brev Launchable install-flow smoke test

3 participants