Walkthrough

Adds a CLI patch that adjusts OpenClaw runtime JS for chat/send run-id correlation and idempotency, integrates the script into Docker and sandbox staging, provides unit and E2E tests (including a cloud-backed TUI correlation harness), extends correlation trace analysis, and wires two selective-dispatch nightly CI jobs into aggregators.

Changes

OpenClaw Chat Correlation Compatibility Layer

Sequence Diagram

sequenceDiagram
  participant Dev as Patch Script
  participant Docker as Docker Build
  participant Dist as OpenClaw dist
  participant Sandbox as Provisioned Sandbox
  participant Vitest as Vitest Live Test
  participant Analysis as Correlation Analysis
  Dev->>Docker: COPY patch-openclaw-chat-send.js into image
  Docker->>Dist: RUN patch script against compiled dist
  Docker->>Sandbox: produce image with patched runtime
  Sandbox->>Vitest: run TUI correlation harness against 2026.5.18
  Vitest->>Sandbox: send prompts via websocket
  Sandbox->>Vitest: stream delta and final chat events
  Vitest->>Analysis: collect and analyze trace events
  Analysis->>Analysis: compute missing/duplicate/correlation metrics
  Vitest->>Vitest: assert metrics empty

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• NVIDIA/NemoClaw#3869: Dockerfile fetch-guard patch extraction and regression test adjustments align with changes to Dockerfile patch sections introduced in that PR.

Suggested labels

enhancement: testing, E2E, Integration: OpenClaw, Docker, NV QA

Suggested reviewers

• jyaunches
• cv

Poem

🐰 I hopped a tiny patch into the Claw tonight,
Run-ids snug and idempotent, replies aligned right,
Sandboxes spin up, Vitest taps the keys,
Traces match prompts and calm the test-day breeze,
Hooray — correlation sleeps tight.

❌ Failed checks (1 warning)

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• openclaw-tui-chat-correlation-e2e (high): Primary required coverage for the new OpenClaw chat.send patch. It builds/onboards a fresh OpenClaw sandbox and runs the live gateway/WebChat correlation harness against rapid sequential sends, validating that the patched behavior is actually baked into the sandbox image.
• cloud-e2e (high): Dockerfile and optimized build-context changes affect the sandbox container image used by normal OpenClaw onboarding. Run the full onboard plus cloud inference path to catch image build, startup, and basic live inference regressions.
• sandbox-survival-e2e (high): The container image change can affect sandbox boot and gateway/runtime resilience. This verifies the sandbox survives gateway restart and continues to serve workspace and inference flows.
• rebuild-openclaw-e2e (high): The PR changes files copied into the sandbox build context and the Dockerfile patch sequence. Rebuild coverage is needed to verify OpenClaw workspace state survives image rebuilds with the new patch asset included.
• hermes-e2e (high): Although the new shim targets OpenClaw, the Dockerfile is a shared sandbox image construction path. Run Hermes onboard and inference to ensure the image build changes do not regress the multi-agent runtime path.

Optional E2E

• token-rotation-e2e (high): Only the E2E script assertions were refactored from pipe grep to here-string grep; no credential runtime logic changed. Useful to validate the edited harness, but not merge-blocking for product behavior.
• issue-3600-gpu-proof-optional-e2e (low): The workflow adds this new selective-dispatch job. There is no corresponding onboard source change in this PR, so it is mainly useful to prove the new workflow job is wired correctly.
• openshell-gateway-upgrade-e2e (high): The workflow changes the setup-node action pin for this job only. Optional smoke of the edited job definition; the PR does not change gateway upgrade runtime code.

New E2E recommendations

• None.

Dispatch hint

• Workflow: nightly-e2e.yaml
• jobs input: openclaw-tui-chat-correlation-e2e,cloud-e2e,sandbox-survival-e2e,rebuild-openclaw-e2e,hermes-e2e

E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required scenario E2E

• None. No scenario workflow, scenario metadata, scenario runtime, or validation-suite files changed.

Optional scenario E2E

• None.

Relevant changed files

• None.

🛠️ Needs attention

• Secret-bearing workflow runs code from target_ref (.github/workflows/nightly-e2e.yaml:407): The OpenClaw TUI/chat job checks out `${{ inputs.target_ref || github.ref }}`, installs dependencies, and runs scripts from that checked-out tree while exposing `NVIDIA_API_KEY` and `GITHUB_TOKEN`. The workflow documents `target_ref` as a way for the trusted main workflow to test a PR head SHA; if that SHA contains untrusted changes, the checked-out `package.json`, npm lifecycle scripts, bash harness, or Vitest tests can read or exfiltrate secrets or abuse checkout credentials. This expands the trusted-code boundary in a high-risk E2E path. This finding was present in the previous advisor review and still applies.
  • Recommendation: Keep workflow and harness code trusted: run the E2E driver from `origin/main` or another reviewed ref, checkout the candidate target into a separate code-under-test directory, disable persisted checkout credentials where not needed, and do not expose repository secrets to scripts sourced from PR-controlled refs. If PR-head execution is required, gate it behind maintainer-only approval and a secret-free mode.
  • Evidence: The job uses `actions/checkout` with `ref: ${{ inputs.target_ref || github.ref }}`; later steps run `npm ci --include=dev` and `bash test/e2e/test-openclaw-tui-chat-correlation.sh` from that checkout with `NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}` and `GITHUB_TOKEN: ${{ github.token }}`.
• [DGX Station][Onboard] cuInit(0) SIGSEGV in verifyDirectSandboxGpu — onboard aborts (v0.0.43 GB300 aarch64) #3600 validation is only a source-order check (.github/workflows/nightly-e2e.yaml:466): The linked [DGX Station][Onboard] cuInit(0) SIGSEGV in verifyDirectSandboxGpu — onboard aborts (v0.0.43 GB300 aarch64) #3600 issue expects onboard to complete on the affected GPU path after an optional direct GPU proof failure. The added `issue-3600-gpu-proof-optional-e2e` job does not run onboard, does not exercise `verifyDirectSandboxGpu`, and does not simulate the optional proof failure path; it only scans source text for one string before another string. That is not sufficient evidence for the issue gate described by the issue and closure comment. This finding was present in the previous advisor review and still applies.
  • Recommendation: Replace or supplement this with a behavioral test that invokes the relevant onboard/GPU-proof path with an optional proof failure and asserts onboard continues without the fatal throw. If GB300 hardware is unavailable, use a focused unit/integration harness around `verifyDirectSandboxGpu` or its caller rather than a string-position test.
  • Evidence: The job reads `src/lib/onboard.ts`, slices from `function verifyDirectSandboxGpu`, and checks that `if (proof.optional === true) return;` appears before `throw new Error(`GPU proof failed:`; it never executes onboard or a verifier command.
• Validation-only changes would become scheduled nightly behavior (.github/workflows/nightly-e2e.yaml:405): The PR description states this is test-evidence-only and not intended to merge into main as-is, but the diff adds production workflow jobs and a Dockerfile OpenClaw runtime patch that would run or ship from main if merged. The new workflow predicates run on scheduled events whenever the job list is empty because the condition allows all non-`workflow_dispatch` events. This finding was present in the previous advisor review and still applies.
  • Recommendation: Before merging to main, either remove the validation-only workflow/Dockerfile wiring or convert it into permanent, reviewed coverage with the trusted-code boundary and acceptance gaps fixed. If the PR is only for evidence collection, keep it out of main and avoid landing the scheduled-job additions.
  • Evidence: The PR body says `not intended to merge into main as-is` and `This PR is for test evidence only.` The diff adds `openclaw-tui-chat-correlation-e2e` and `issue-3600-gpu-proof-optional-e2e` under the nightly workflow with predicates that run for non-dispatch events, and Dockerfile now executes `node /usr/local/lib/nemoclaw/patch-openclaw-chat-send.js` during sandbox image builds.

🔎 Worth checking

• Nightly E2E hard-codes a specific OpenClaw version (test/e2e/test-openclaw-tui-chat-correlation.sh:47): The new nightly wrapper requires the sandbox OpenClaw version output to contain `2026.5.18`. If this lands as a scheduled nightly job, normal OpenClaw updates can break the validation for reasons unrelated to TUI/chat correlation, and the test may fail before reaching the regression assertions. This finding was present in the previous advisor review and still applies.
  • Recommendation: Avoid hard-coding a single release version in a scheduled nightly test, or make the expected version an explicit workflow input/environment variable for one-off validation. For permanent coverage, assert required capabilities or protocol behavior instead of an exact version string.
  • Evidence: The script runs `openshell sandbox exec --name "$SANDBOX_NAME" -- openclaw --version` and exits unless `grep -q "2026.5.18"` succeeds.

🌱 Nice ideas

• None.

Current findings:

• Secret-bearing workflow runs code from target_ref (.github/workflows/nightly-e2e.yaml:407): The OpenClaw TUI/chat job checks out `${{ inputs.target_ref || github.ref }}`, installs dependencies, and runs scripts from that checked-out tree while exposing `NVIDIA_API_KEY` and `GITHUB_TOKEN`. The workflow documents `target_ref` as a way for the trusted main workflow to test a PR head SHA; if that SHA contains untrusted changes, the checked-out `package.json`, npm lifecycle scripts, bash harness, or Vitest tests can read or exfiltrate secrets or abuse checkout credentials. This expands the trusted-code boundary in a high-risk E2E path. This finding was present in the previous advisor review and still applies.
  • Recommendation: Keep workflow and harness code trusted: run the E2E driver from `origin/main` or another reviewed ref, checkout the candidate target into a separate code-under-test directory, disable persisted checkout credentials where not needed, and do not expose repository secrets to scripts sourced from PR-controlled refs. If PR-head execution is required, gate it behind maintainer-only approval and a secret-free mode.
  • Evidence: The job uses `actions/checkout` with `ref: ${{ inputs.target_ref || github.ref }}`; later steps run `npm ci --include=dev` and `bash test/e2e/test-openclaw-tui-chat-correlation.sh` from that checkout with `NVIDIA_API_KEY: ${{ secrets.NVIDIA_API_KEY }}` and `GITHUB_TOKEN: ${{ github.token }}`.
• [DGX Station][Onboard] cuInit(0) SIGSEGV in verifyDirectSandboxGpu — onboard aborts (v0.0.43 GB300 aarch64) #3600 validation is only a source-order check (.github/workflows/nightly-e2e.yaml:466): The linked [DGX Station][Onboard] cuInit(0) SIGSEGV in verifyDirectSandboxGpu — onboard aborts (v0.0.43 GB300 aarch64) #3600 issue expects onboard to complete on the affected GPU path after an optional direct GPU proof failure. The added `issue-3600-gpu-proof-optional-e2e` job does not run onboard, does not exercise `verifyDirectSandboxGpu`, and does not simulate the optional proof failure path; it only scans source text for one string before another string. That is not sufficient evidence for the issue gate described by the issue and closure comment. This finding was present in the previous advisor review and still applies.
  • Recommendation: Replace or supplement this with a behavioral test that invokes the relevant onboard/GPU-proof path with an optional proof failure and asserts onboard continues without the fatal throw. If GB300 hardware is unavailable, use a focused unit/integration harness around `verifyDirectSandboxGpu` or its caller rather than a string-position test.
  • Evidence: The job reads `src/lib/onboard.ts`, slices from `function verifyDirectSandboxGpu`, and checks that `if (proof.optional === true) return;` appears before `throw new Error(`GPU proof failed:`; it never executes onboard or a verifier command.
• Validation-only changes would become scheduled nightly behavior (.github/workflows/nightly-e2e.yaml:405): The PR description states this is test-evidence-only and not intended to merge into main as-is, but the diff adds production workflow jobs and a Dockerfile OpenClaw runtime patch that would run or ship from main if merged. The new workflow predicates run on scheduled events whenever the job list is empty because the condition allows all non-`workflow_dispatch` events. This finding was present in the previous advisor review and still applies.
  • Recommendation: Before merging to main, either remove the validation-only workflow/Dockerfile wiring or convert it into permanent, reviewed coverage with the trusted-code boundary and acceptance gaps fixed. If the PR is only for evidence collection, keep it out of main and avoid landing the scheduled-job additions.
  • Evidence: The PR body says `not intended to merge into main as-is` and `This PR is for test evidence only.` The diff adds `openclaw-tui-chat-correlation-e2e` and `issue-3600-gpu-proof-optional-e2e` under the nightly workflow with predicates that run for non-dispatch events, and Dockerfile now executes `node /usr/local/lib/nemoclaw/patch-openclaw-chat-send.js` during sandbox image builds.
• Nightly E2E hard-codes a specific OpenClaw version (test/e2e/test-openclaw-tui-chat-correlation.sh:47): The new nightly wrapper requires the sandbox OpenClaw version output to contain `2026.5.18`. If this lands as a scheduled nightly job, normal OpenClaw updates can break the validation for reasons unrelated to TUI/chat correlation, and the test may fail before reaching the regression assertions. This finding was present in the previous advisor review and still applies.
  • Recommendation: Avoid hard-coding a single release version in a scheduled nightly test, or make the expected version an explicit workflow input/environment variable for one-off validation. For permanent coverage, assert required capabilities or protocol behavior instead of an exact version string.
  • Evidence: The script runs `openshell sandbox exec --name "$SANDBOX_NAME" -- openclaw --version` and exits unless `grep -q "2026.5.18"` succeeds.