test(e2e): migrate test-launchable-smoke.sh to vitest [ANCHOR-8]#5219
Merged
jyaunches merged 9 commits intoJun 11, 2026
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a live Vitest "launchable smoke" test and a CI job. The test runs the Ubuntu launchable install script, onboards a sandbox, verifies direct and sandbox-hosted inference (expects "PONG"), runs an OpenClaw agent interaction (expects "42"), and performs teardown and artifact upload. Workflow gains a ChangesLaunchable E2E Smoke Test
Sequence Diagram(s)sequenceDiagram
participant User as Workflow Trigger
participant Actions as GitHub Actions
participant Runner as GitHub Runner
participant Vitest as launchable-smoke.test.ts
participant Sandbox as NemoClaw Sandbox
participant Artifacts as Artifact Storage
User->>Actions: workflow_dispatch (inputs.jobs)
Actions->>Runner: start launchable-smoke-vitest job
Runner->>Vitest: run test/e2e-scenario/live/launchable-smoke.test.ts
Vitest->>Sandbox: nemoclaw onboard / run sandboxed inference / openclaw agent
Sandbox-->>Vitest: inference responses (PONG) and agent output (JSON containing 42)
Vitest->>Artifacts: upload e2e-artifacts/vitest/launchable-smoke/
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e-scenario/live/launchable-smoke.test.ts`:
- Around line 361-369: The gateway-container check currently uses "grep -E
'nemoclaw|openshell' || true" so runBash always succeeds and the test never
fails; update the runBash call (or its command) so it returns a non-zero exit
when no matching container is found (remove the "|| true" and/or use grep -q and
let runBash throw), then after capturing gatewayContainer, assert or throw if
gatewayContainer.stdout.trim().length === 0 (or set confirmed = false and call
throw/new Error) before writing artifacts; refer to the runBash invocation and
the gatewayContainer variable and artifacts.writeJson to locate where to
implement this failure behavior.
- Around line 28-34: The call to validateSandboxName(SANDBOX_NAME) runs at
module evaluation and should be moved inside the live-test gate so it only runs
when shouldRunLiveE2EScenarios() returns true; remove the top-level
validateSandboxName invocation and add a call to
validateSandboxName(SANDBOX_NAME) at the start of the gated block that checks
shouldRunLiveE2EScenarios(), ensuring SANDBOX_NAME remains declared as before
and that any subsequent code in that gated branch uses the validated value.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 7e0fcd2c-2771-4f3b-a64e-1c9306c2506f
📒 Files selected for processing (1)
test/e2e-scenario/live/launchable-smoke.test.ts
Contributor
Author
|
/nvskills-ci |
11730c7 to
10dde54
Compare
10dde54 to
e78bb73
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e-scenario/live/launchable-smoke.test.ts`:
- Line 28: The SANDBOX_NAME constant currently falls back to a static
"e2e-launchable" causing cross-run collisions; modify the initialization of
SANDBOX_NAME so that when process.env.NEMOCLAW_SANDBOX_NAME is unset you
generate a single per-process unique default (e.g., compute a uniqueSuffix once
using crypto.randomUUID() or Date.now()+Math.random and set SANDBOX_NAME =
`e2e-launchable-${uniqueSuffix}`) and reuse that constant for provisioning and
teardown so the same run uses the same name while explicit env overrides remain
unchanged.
- Around line 413-433: The test currently parses agent.stdout and checks for
"42" even if the process failed; update the test to assert the subprocess
succeeded first by checking agent.exitCode === 0 (e.g.
expect(agent.exitCode).toBe(0) with a helpful message including
stdout/stderr/rc) immediately after calling sandboxExec and before calling
parseAgentText; then proceed to parseAgentText(agent.stdout) and the existing
regex assertion. Target the sandboxExec result variable "agent" and the parse
function "parseAgentText".
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2745707e-09ff-4f6c-9690-d918b3e90c4e
📒 Files selected for processing (2)
.github/workflows/e2e-vitest-scenarios.yamltest/e2e-scenario/live/launchable-smoke.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- .github/workflows/e2e-vitest-scenarios.yaml
e78bb73 to
14b38bd
Compare
…chable-smoke # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml
…chable-smoke # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 27377259096
|
cv
approved these changes
Jun 11, 2026
…chable-smoke # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 27379874314
|
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27380320994
|
…chable-smoke # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27380701353
|
79 tasks
This was referenced Jun 12, 2026
This was referenced Jun 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrate
test/e2e/test-launchable-smoke.shwith the simplest equivalent live Vitest coverage.Related Issues
Refs #5098
Refs #2599
Contract mapping
scripts/brev-launchable-ci-cpu.shagainst the current checkout via a pre-seeded clone.test/e2e-scenario/live/launchable-smoke.test.tsrunssudo -E bash scripts/brev-launchable-ci-cpu.shwithNEMOCLAW_CLONE_DIRpointed at a seeded local clone.sudo, Docker, Node/OpenShell install checks, sentinel file, repo/plugin build artifacts.nemoclaw onboard --non-interactivecreates a cloud OpenClaw sandbox.nemoclawfrom the clone directory withNVIDIA_API_KEY,NEMOCLAW_SANDBOX_NAME,NEMOCLAW_RECREATE_SANDBOX=1, and the workflow-provided live model.nemoclaw list,nemoclaw status,openshell inference get, and gateway container evidence.inference.local, andopenclaw agent --json --thinking offproduce expected model responses.Simplicity check
.github/workflows/nightly-e2e.yamljoblaunchable-smoke-e2eonubuntu-latest, requiring sudo, Docker/OpenShell, andNVIDIA_API_KEY..github/workflows/e2e-vitest-scenarios.yamljoblaunchable-smoke-vitestonubuntu-latestwith the same sudo/Docker/OpenShell/NVIDIA_API_KEYresource class.launchable-smoke-vitestjob ine2e-vitest-scenarios.yaml; legacy shell script deletion and nightly shell lane retirement are deferred to Epic: Migrate legacy bash E2E into the Vitest E2E system #5098 Phase 11.gh workflow run e2e-vitest-scenarios.yaml --repo NVIDIA/NemoClaw --ref e2e-migrate/test-launchable-smoke -f jobs=launchable-smoke-vitest -f pr_number=5219.Verification
NEMOCLAW_RUN_E2E_SCENARIOS=1 npx vitest run --project e2e-scenarios-live test/e2e-scenario/live/launchable-smoke.test.ts --silent=false --reporter=default(local run skips without live secret/environment)npx vitest run --project e2e-vitest-support test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts --silent=false --reporter=defaultnpx @biomejs/biome lint test/e2e-scenario/live/launchable-smoke.test.tsnpm run test-size:checkgit diff --checklaunchable-smoke-vitestpassed onubuntu-latestat head90a7e0a63f28ae76225d0828711b8815457c6994.