test(e2e): migrate OpenClaw inference switch scenario#5357
Merged
Conversation
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a Vitest live E2E scenario that migrates the legacy openclaw inference-switch script into TypeScript tests (including a mock Anthropic provider, resilient inference/agent checks, retries, cleanup) and wires a new conditional GitHub Actions job to run the test and report results. ChangesOpenClaw Inference Switch E2E Test Migration
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Contributor
E2E Advisor RecommendationRequired E2E: None Full advisor summaryE2E Recommendation AdvisorFailed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-advisor-raw-output.txt |
Contributor
Vitest E2E Scenario RecommendationRequired Vitest E2E scenarios: None Full Vitest E2E advisor summaryVitest E2E Scenario AdvisorFailed: Could not parse JSON from advisor output; see /home/runner/work/NemoClaw/NemoClaw/artifacts/e2e-advisor/e2e-scenario-advisor-raw-output.txt |
Contributor
PR Review AdvisorFindings: 0 needs attention, 1 worth checking, 0 nice ideas Review findings🛠️ Needs attention
🔎 Worth checking
🌱 Nice ideas
Consider writing more tests for
This is an automated advisory review. A human maintainer must make the final merge decision. |
Contributor
Vitest E2E Scenario Results — ❌ Some jobs failedRun: 27436819548
|
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27437824573
|
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27438779472
|
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/e2e-scenario/live/openclaw-inference-switch.test.ts`:
- Around line 34-39: The test currently uses fixed defaults for SANDBOX_NAME and
SWITCH_MOCK_PORT which cause collisions; update the initialization to generate
unique sandbox names and use a validated port parser that accepts 0
(auto-assign) for SWITCH_MOCK_PORT (e.g., implement a parsePortEnv(name,
fallback) that validates integer 0–65535 and returns Number.parseInt or
fallback), and replace direct uses of SWITCH_MOCK_PORT and SANDBOX_NAME with
these new validated values so startMockAnthropicProvider() can read the bound
port and cleanup/pre-final steps use the unique sandbox name.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2cd3681f-b72d-42c7-a6ea-e0cb27eaf8e3
📒 Files selected for processing (4)
.github/workflows/e2e-vitest-scenarios.yamltest/e2e-scenario/live/openclaw-inference-switch.test.tstest/e2e-scenario/support-tests/e2e-scenarios-workflow.test.tstools/e2e-scenarios/free-standing-jobs.env
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
Vitest E2E Scenario Results — ✅ All jobs passedRun: 27446110743
|
…nference-switch # Conflicts: # .github/workflows/e2e-vitest-scenarios.yaml # test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tools/e2e-scenarios/workflow-boundary.mts (1)
2543-2543: ⚡ Quick winAdd full boundary assertions for
openclaw-inference-switch-vitest, not just selector mapping.Line 2543 currently validates only the
ifselector contract. Please add a dedicated validator (like other live free-standing jobs) to enforce required env values, secret scoping, run command target, artifact paths, timeout, and cleanup step behavior; otherwise workflow drift in this job can slip past boundary tests.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/e2e-scenarios/workflow-boundary.mts` at line 2543, The selector-only check for openclaw-inference-switch-vitest is insufficient; add a full boundary validator alongside validateFreeStandingJobSelector by creating/invoking the same per-job validator used for other live free-standing jobs (e.g., validateFreeStandingJob or validateFreeStandingJobBoundary) for "openclaw-inference-switch-vitest" and assert the required env vars, scoped secrets, expected run command target, artifact input/output paths, timeout value, and presence/behavior of the cleanup step so this job cannot drift undetected; reference the existing validateFreeStandingJobSelector call and mirror the validation pattern and assertions used for other jobs in this file when adding the new validator invocation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tools/e2e-scenarios/workflow-boundary.mts`:
- Line 2543: The selector-only check for openclaw-inference-switch-vitest is
insufficient; add a full boundary validator alongside
validateFreeStandingJobSelector by creating/invoking the same per-job validator
used for other live free-standing jobs (e.g., validateFreeStandingJob or
validateFreeStandingJobBoundary) for "openclaw-inference-switch-vitest" and
assert the required env vars, scoped secrets, expected run command target,
artifact input/output paths, timeout value, and presence/behavior of the cleanup
step so this job cannot drift undetected; reference the existing
validateFreeStandingJobSelector call and mirror the validation pattern and
assertions used for other jobs in this file when adding the new validator
invocation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 52253fa8-e86d-4aaf-b493-84f364b3ddb2
📒 Files selected for processing (3)
.github/workflows/e2e-vitest-scenarios.yamltest/e2e-scenario/support-tests/e2e-scenarios-workflow.test.tstools/e2e-scenarios/workflow-boundary.mts
✅ Files skipped from review due to trivial changes (1)
- test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrates
test/e2e/test-openclaw-inference-switch.shto a focused live Vitest scenario while preserving the real install.sh/OpenShell/Docker/inference switch boundary. Adds manual workflow dispatch wiring so the free-standing scenario can run throughe2e-vitest-scenarios.Related Issue
Fixes #5098
Changes
test/e2e-scenario/live/openclaw-inference-switch.test.tsto cover install, inference route switching, OpenClaw config/hash, registry/session state,inference.local, and OpenClaw agent PONG checks.openclaw-inference-switchin the free-standing Vitest scenario inventory and workflow dispatch job.Type of Change
Verification
npx prek run --all-filespassesnpm testpassesnpm run docsbuilds without warnings (doc changes only)Signed-off-by: Carlos Villela cvillela@nvidia.com
Summary by CodeRabbit