Skip to content

nightly-e2e: agent reply JSON extraction broken by OpenClaw schema change (5 jobs) #4031

@hunglp6d

Description

@hunglp6d

Nightly E2E failure: agent reply JSON extraction broken by OpenClaw schema change

Run: https://github.com/NVIDIA/NemoClaw/actions/runs/26260886472
Date: 2026-05-22
Commit: 74c0246
Affected jobs (5):

Bug group: agent-reply-json-extract
Failure class: config_error
Confidence: high

Root cause

OpenClaw ≥ 2026.5.18 changed its openclaw agent --json output from nesting payloads under {"result": {"payloads": [...]}} to returning them at the top level {"payloads": [...]}. Five E2E test scripts use inline Python extractors that hardcode doc.get("result") or {}, which returns {} under the new schema — so .get("payloads") always yields [] → empty reply → assertion failure.

Evidence

All 5 jobs show the same pattern:

  • Raw agent output contains correct payload text (e.g., "42", "PONG") at top level
  • Python extractor returns reply='' because it looks only in doc["result"]
  • Assertion fails on empty reply

Fix

PR: #4030

The one-line fix in each script changes doc.get("result") or {}doc.get("result") or doc, which falls back to the document root when the result wrapper is absent. Backwards-compatible with the old schema.

Files changed

  • test/e2e/test-bedrock-runtime-compatible-anthropic.sh (line 713)
  • test/e2e/test-launchable-smoke.sh (line 531)
  • test/e2e/test-messaging-compatible-endpoint.sh (line 534)
  • test/e2e/test-openclaw-inference-switch.sh (line 284)
  • test/e2e/test-sandbox-operations.sh (line 393)

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: e2eEnd-to-end tests, nightly failures, or validation infrastructure

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions