docs(e2e): clarify single-runner scenario target by cv · Pull Request #4939 · NVIDIA/NemoClaw

cv · 2026-06-08T06:27:53Z

Summary

Clarifies that the current E2E scenario hybrid model is transitional, not the intended end state for #3588. Documents the single-runner target and gives maintainers a rule for treating YAML/bash runner work as bridge work rather than durable architecture.

Related Issue

Refs #3588.

Changes

Added a target runner model to test/e2e-scenario/docs/README.md.
Updated migration notes to prefer single-runner behavior and treat YAML/bash updates as bridge work.
Clarified that Phase 1: Environment, Manifest, Fixture, and Runtime Action Primitives (E2E audit-coverage) #4347-Phase 11: Final Audit Reconciliation (E2E audit-coverage) #4357 are acceptance coverage requirements, not permanent YAML/bash deliverable requirements.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Verification

npx prek run --all-files passes
npm test passes
Tests added or updated for new or changed behavior
No secrets, API keys, or credentials committed
Docs updated for user-facing behavior changes
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Additional verification:

npm run docs completed successfully, but Fern reported 2 warnings, so the no-warnings checkbox is intentionally left unchecked.

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

Documentation
- Enhanced E2E scenario migration documentation with clarified target architecture, revised migration steps with tighter criteria, and explicit cleanup rules for contributors.
- Updated framework overview documentation to describe the desired single-runner architecture properties, identify authoritative surfaces, and provide near-term alignment guidance during the transition period.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

coderabbitai · 2026-06-08T06:28:06Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: edadb0b9-a659-452e-bcf3-0f4d7b6c048a

📥 Commits

Reviewing files that changed from the base of the PR and between ec408c8 and 4a4bf4b.

📒 Files selected for processing (2)

test/e2e-scenario/docs/MIGRATION.md
test/e2e-scenario/docs/README.md

📝 Walkthrough

Walkthrough

This PR updates E2E scenario migration documentation across two files to clarify the hybrid state as transitional, define a target single-runner architecture, specify current authoritative surfaces during migration, establish execution guidance for contributors, and add explicit cleanup rules.

Changes

E2E Migration Documentation Clarifications

Layer / File(s)	Summary
Target architecture and framework state clarification `test/e2e-scenario/docs/README.md`, `test/e2e-scenario/docs/MIGRATION.md`	Clarifies the E2E scenario framework as transitional, states the target as a single scenario runner owning resolution/orchestration/evidence/redaction/assertion dispatch, and documents the target architecture with runner ownership model (workflow, assertions/evidence, secret redaction, shell boundaries).
Current sources of truth and migration boundaries `test/e2e-scenario/docs/README.md`	Establishes which repository surfaces (typed and shell) are authoritative during migration, discourages new legacy `test/e2e/test-*.sh` entrypoints, updates the YAML/shell runner conceptual mapping, and clarifies that the typed scenario registry drives plan artifacts toward the single-runner model.
Target runner model properties and capabilities `test/e2e-scenario/docs/README.md`	Expands desired runner properties with typed plan as source of truth, declarative manifests, TypeScript probe preference, controlled shell spawn boundaries, bridging-plan obligations, and audit-phase acceptance requirements tied to single-runner equivalents or maintainer-approved bridge paths.
Migration execution steps and cleanup rules `test/e2e-scenario/docs/MIGRATION.md`	Tightens the "What to migrate next" checklist (items 4–8) with stricter YAML metadata update criteria, emphasizes assertion/probe and helper framework coverage before leaving legacy scripts, ties retention to `#4357` deletion readiness, and adds cleanup rules preferring single-runner behavior with documented bridge-change obligations.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

NVIDIA/NemoClaw#4654: Concurrently updates the same E2E migration documentation files to refine the hybrid-to-single-runner migration model, cleanup rules, and contributor guidance.
NVIDIA/NemoClaw#4649: Updates the same E2E migration documentation to shift migration-state policy from repo-local tracking toward issue/PR-managed ownership, providing policy context for these documentation clarifications.

Suggested labels

area: docs, area: e2e

Suggested reviewers

cjagwani
prekshivyas

Poem

🐰 In hybrid lands where runners roam,
We chart the path back to one home—
Clear targets guide the migration's way,
With cleanup rules to light the day,
For single runners yet to stay.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'docs(e2e): clarify single-runner scenario target' directly and precisely matches the PR's main change: documentation updates that clarify the single-runner scenario as the intended target architecture for the E2E framework.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/e2e-single-runner-direction

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-06-08T06:28:34Z

E2E Advisor Recommendation

Required E2E: None
Optional E2E: None

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

None. No E2E required: this PR changes documentation only under test/e2e-scenario/docs and cannot affect installer/onboarding, sandbox lifecycle, credentials, security boundaries, network policy, inference routing, deployment, or real assistant user flows.

Optional E2E

None.

New E2E recommendations

None.

github-actions · 2026-06-08T06:28:34Z

E2E Scenario Advisor Recommendation

Required scenario E2E: None
Optional scenario E2E: None

Workflow run

Full scenario advisor summary

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

None. Docs-only changes under test/e2e-scenario/docs do not affect scenario runtime, workflow routing, metadata catalogs, expected-state contracts, suite definitions, or executable scenario behavior.

Optional scenario E2E

None.

Relevant changed files

test/e2e-scenario/docs/MIGRATION.md
test/e2e-scenario/docs/README.md

github-actions · 2026-06-08T06:30:24Z

PR Review Advisor

Findings: 0 needs attention, 0 worth checking, 0 nice ideas
Since last review: 0 prior items resolved, 0 still apply, 0 new items found

Consider writing more tests for

**Acceptance clause:** Refs Implement layered E2E scenario model #3588. — add test evidence or identify existing coverage. The deterministic context did not include Implement layered E2E scenario model #3588 issue body or comments, so no literal issue acceptance clauses could be verified beyond the PR's own reference and the diff's Implement layered E2E scenario model #3588 documentation updates.

Workflow run details

This is an automated advisory review. A human maintainer must make the final merge decision.

wscurran · 2026-06-08T14:04:08Z

✨
Related open issues:

jyaunches

Yeah, agree with & like this direction

#4939 added 'transitional model / target runner' guidance to docs/README.md after our merge. Resolved by taking that content but reconciling it with this PR's retirement of nemoclaw_scenarios/expected-states.yaml and the runtime/resolver/ tree: - 'Current sources of truth' table: drop the row pointing at expected-states.yaml and runtime/resolver; add a row for the typed expected-state registry (scenarios/expected-states.ts) as the single source of truth. - Replaced 'fan-out and dry-run planning' with 'fan-out and live execution'. - Added one paragraph noting the legacy YAML resolver and expected-states.yaml have been retired so the doc and the codebase agree.

docs(e2e): clarify single-runner scenario target

4a4bf4b

Signed-off-by: Carlos Villela <cvillela@nvidia.com>

cv self-assigned this Jun 8, 2026

This was referenced Jun 8, 2026

test(e2e): run onboarding assertions in scenario runner #4657

Closed

Implement layered E2E scenario model #3588

Closed

cv requested a review from jyaunches June 8, 2026 06:30

cv mentioned this pull request Jun 8, 2026

Adopt Vitest fixtures as the E2E scenario execution model #4941

Open

7 tasks

wscurran added area: docs Documentation, examples, guides, or docs build area: e2e End-to-end tests, nightly failures, or validation infrastructure labels Jun 8, 2026

jyaunches approved these changes Jun 8, 2026

View reviewed changes

Merge branch 'main' into codex/e2e-single-runner-direction

cca267c

cv merged commit 2c5283c into main Jun 8, 2026
34 checks passed

cv deleted the codex/e2e-single-runner-direction branch June 8, 2026 15:59

coderabbitai Bot mentioned this pull request Jun 8, 2026

docs(e2e): choose Vitest fixtures as scenario runner #4963

Merged

12 tasks

cv added the v0.0.61 Release target label Jun 8, 2026

wscurran added the chore Build, CI, dependency, or tooling maintenance label Jun 8, 2026

coderabbitai Bot mentioned this pull request Jun 9, 2026

chore(e2e): refresh docs and add real gateway negative-path test #5042

Merged

12 tasks

Conversation

cv commented Jun 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Type of Change

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested labels

Suggested reviewers

Poem

Uh oh!

github-actions Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Uh oh!

github-actions Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E Scenario Advisor Recommendation

E2E Scenario Advisor

Required scenario E2E

Optional scenario E2E

Relevant changed files

Uh oh!

github-actions Bot commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor

Uh oh!

wscurran commented Jun 8, 2026

Uh oh!

jyaunches left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cv commented Jun 8, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 8, 2026 •

edited

Loading

github-actions Bot commented Jun 8, 2026 •

edited

Loading

github-actions Bot commented Jun 8, 2026 •

edited

Loading

github-actions Bot commented Jun 8, 2026 •

edited

Loading