Skip to content

[QA harness] mock Pi provider id and apply_patch failure fixture distort parity #80320

@100yenadmin

Description

@100yenadmin

Correction TLDR

Status: QA harness bug, not a Codex user-facing apply_patch bug.

The original report conflated two harness problems with product behavior:

  1. Provider/profile setup bug: the Pi cell stayed on mock-openai/<model>, but Pi only exposes apply_patch for OpenAI-compatible provider ids such as openai / openai-codex. The mock server already clones the endpoint under openai, so the parity harness needed to remap forced runtime cells correctly.
  2. Fixture fault-injection boundary: synthetic failure-path inputs must be treated as QA fixture/fault-injection behavior, not proof that Codex users lose patching.

Product impact if OpenClaw moved fully to Codex today: P4. Product patching is not proven broken.

QA impact: resolved for the beta.5 mock hard gate. The corrected tool-defaults direct lane now passes 20/20 with zero skips and zero failures.

Latest Beta.5 Evidence

OpenClaw baseline: v2026.5.10-beta.5
PR: #80323
PR head: 3336dec6419c9cc9a87dc7cfa6f48118ca2d838e
Remote proof run: https://github.com/electricsheephq/openclaw-local-test/actions/runs/25719383976
Confidence tracker: #80936

Relevant artifact result:

{
  "tool-defaults-direct": { "total": 20, "passed": 20, "skipped": 0, "failed": 0 },
  "confidence-report": { "pass": true, "zeroUnknowns": true }
}

Correct Fix

  • Remap mock runtime-pair provider/model ids so the Pi and Codex cells exercise the intended OpenAI-compatible tool surface.
  • Keep apply_patch in default-tool coverage.
  • Do not file this as a product/Codex runtime bug unless a live/native Codex run reproduces patch failure outside this mock fixture.

Superseded Original Report

The earlier wording described this as Pi omitting apply_patch while Codex exposed it. That was true only inside an invalid QA mock provider setup and should not be read as a user-facing runtime regression.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions