test(e2e): add OpenShell version pin guard for #3474 by jyaunches · Pull Request #3478 · NVIDIA/NemoClaw

jyaunches · 2026-05-13T20:17:50Z

Summary

Adds a failing-test-first regression guard for #3474 / Margaret's QA report:

simulates a host where openshell --version reports 0.0.40 while NemoClaw's installer supports max 0.0.39
asserts scripts/install-openshell.sh should recover by downloading/replacing with pinned OpenShell v0.0.39
wires the guard into regression-e2e.yaml as openshell-version-pin-e2e

Failing-test-first note

This test is expected to fail on current main until the fix lands. Current main exits early with:

openshell 0.0.40 is above the maximum (0.0.39) supported by this NemoClaw release. Upgrade NemoClaw first.

The desired fixed behavior is that the installer warns/reinstalls the pinned compatible OpenShell release instead of hard-failing on sticky openshell 0.0.40.

Regression holding pen

This job lives in .github/workflows/regression-e2e.yaml and is not part of scheduled nightly E2E. It can be dispatched explicitly with:

gh workflow run regression-e2e.yaml -f jobs=openshell-version-pin-e2e --ref test/openshell-version-pin-e2e-guard

After the fix PR lands, this guard can be reviewed for promotion or left as an explicit regression dispatch.

Related: #3474

Summary by CodeRabbit

Tests
- Added end-to-end regression test for OpenShell version pinning. This new test creates a simulated host environment and validates that the installer correctly identifies and handles version conflicts when a newer OpenShell version is already installed, ensuring it properly downgrades to the required pinned version during installation.

Adds a failing regression E2E guard for the sticky OpenShell case reported in #3474. Until the fix lands, the targeted regression openshell-version-pin-e2e dispatch will fail because install-openshell.sh hard-fails when an existing openshell 0.0.40 is above NemoClaw's max supported 0.0.39 instead of reinstalling the pinned compatible release. Related: #3474

coderabbitai · 2026-05-13T20:18:08Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cb01dcc8-bbb1-480c-aaaf-4afaf63bca23

📥 Commits

Reviewing files that changed from the base of the PR and between 11837d8 and b199315.

📒 Files selected for processing (2)

.github/workflows/regression-e2e.yaml
test/e2e/test-openshell-version-pin.sh

📝 Walkthrough

Walkthrough

This PR adds end-to-end regression coverage for OpenShell version pinning. The workflow is extended to include a new selectable openshell-version-pin-e2e regression job, and a corresponding test script validates that the installer correctly handles scenarios where a too-new OpenShell version is already installed by downgrading to the pinned compatible version.

Changes

OpenShell Version Pin E2E Regression Test

Layer / File(s)	Summary
Workflow regression job configuration `.github/workflows/regression-e2e.yaml`	Registers `openshell-version-pin-e2e` as a valid dispatch job, adds `openshell_version_pin` output to `select_regression_jobs`, derives that output from the job request list, and defines the new job with repository gating and failure artifact uploads.
E2E test script and hermetic environment `test/e2e/test-openshell-version-pin.sh`	Implements a regression test using a hermetic fake toolchain that simulates a host with pre-installed OpenShell 0.0.40, runs the installer, and validates it downgrades to the pinned 0.0.39 version without failure.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

NVIDIA/NemoClaw#3474: This PR directly addresses the OpenShell version mismatch issue by adding an E2E regression test that reproduces and validates the version pinning behavior.

Possibly related PRs

NVIDIA/NemoClaw#3411: Both PRs extend the .github/workflows/regression-e2e.yaml workflow's select_regression_jobs output and dispatch job selection logic to register new regression E2E jobs.

Suggested labels

enhancement: testing, E2E, v0.0.40

Suggested reviewers

cv
ericksoa

Poem

🐰 A version that's new causes trouble and stress,
So we pin what we need—nothing less, nothing less!
With tests locked down tight in a hermetic shell,
The installer descends when versions don't gel. ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'test(e2e): add OpenShell version pin guard for `#3474`' clearly and concisely summarizes the main change—adding a new end-to-end test for OpenShell version pinning—and directly references the related issue.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch test/openshell-version-pin-e2e-guard

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-05-13T20:19:11Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: openshell-version-pin-e2e, gateway-health-honest-e2e, gateway-drift-preflight-e2e

Dispatch hint: openshell-version-pin-e2e

Workflow run

Full advisor summary

Pi Semantic E2E Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. This PR is purely E2E test infrastructure: it wires a new failing-test-first coverage guard (test-openshell-version-pin.sh) into the workflow_dispatch-only regression-e2e workflow and does not touch product code, scripts/install-openshell.sh, src/, or any user-facing flow. No nightly, branch-validation, sandbox, inference-routing, networking, credential, or deployment surface is modified, so no required E2E run is needed to land the change. The new openshell-version-pin-e2e job and its sibling regression jobs are listed as optional dispatches to verify wiring and selector-logic integrity.

Optional E2E

openshell-version-pin-e2e (low): Newly added regression job. Worth dispatching once via regression-e2e to validate the workflow wiring (job selector, log upload paths, script PATH/shim setup) even though the test is expected to fail on main until the [All Platforms][Install] OpenShell 0.0.40 exceeds NemoClaw v0.0.40 max 0.0.39 — installer should pin compatible OpenShell #3474 fix lands. Not merge-blocking because the PR adds test infra only and the comment in regression-e2e.yaml explicitly states these jobs are holding-pen, failing-test-first guards.
gateway-health-honest-e2e (low): Sibling regression job in the same workflow. Dispatching it alongside the new job confirms the shared select_regression_jobs selector logic still routes correctly after the new openshell_version_pin output was added.
gateway-drift-preflight-e2e (low): Other sibling regression job. Optional smoke to confirm the selector refactor (added openshell_version_pin output) did not regress existing dispatch routing.

New E2E recommendations

None.

Dispatch hint

Workflow: regression-e2e.yaml
jobs input: openshell-version-pin-e2e

cv approved these changes May 13, 2026

View reviewed changes

cv enabled auto-merge (squash) May 13, 2026 20:20

cv merged commit 1da8502 into main May 13, 2026
29 of 30 checks passed

This was referenced May 13, 2026

fix(onboard): pin openshell fetch to blueprint max_openshell_version (#3404) #3446

Merged

[All Platforms][Install] OpenShell 0.0.40 exceeds NemoClaw v0.0.40 max 0.0.39 — installer should pin compatible OpenShell #3474

Closed

cjagwani mentioned this pull request May 14, 2026

0.0.35 OpenShell compat: supervisor↔gateway wire change breaks existing NemoClaw sandboxes on cluster upgrade #2291

Closed

wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026

jyaunches deleted the test/openshell-version-pin-e2e-guard branch June 12, 2026 13:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(e2e): add OpenShell version pin guard for #3474#3478

test(e2e): add OpenShell version pin guard for #3474#3478
cv merged 1 commit into
mainfrom
test/openshell-version-pin-e2e-guard

jyaunches commented May 13, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 13, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Possibly related issues

Possibly related PRs

Suggested labels

Suggested reviewers

Poem

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 13, 2026

Pi Semantic E2E Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jyaunches commented May 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Failing-test-first note

Regression holding pen

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related issues

Possibly related PRs

Suggested labels

Suggested reviewers

Poem

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented May 13, 2026

E2E Advisor Recommendation

Pi Semantic E2E Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jyaunches commented May 13, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 13, 2026 •

edited

Loading