Fix/codex deactivated workspace failover by litang9 · Pull Request #55893 · openclaw/openclaw

litang9 · 2026-03-27T16:12:45Z

Summary

Problem: Codex deactivated_workspace was not classified as a failover-eligible permanent auth condition, so OpenClaw could stop on an unclassified provider failure instead of entering the existing fallback path.
What changed: added a narrow deactivated[_ ]workspace matcher to HIGH_CONFIDENCE_AUTH_PERMANENT_PATTERNS and covered it at helper and failover-reason boundaries.
Scope boundary: this does not change fallback ordering, cooldown policy, auth storage, provider login, active workspace selection, or network behavior.
Related issue: OpenAI Codex OAuth can bind to a deactivated ChatGPT workspace when accounts have multiple workspaces #56693 tracks the broader active-workspace selection/recovery problem; this PR is only the classifier mitigation.

Change Type

Root Cause / Regression History

The Codex provider can return {"detail":{"code":"deactivated_workspace"}} for a deactivated workspace, but the failover classifier only recognized revoked/deactivated API keys, disabled/revoked keys, deactivated accounts, and organization blocks. The missing marker meant resolveFailoverReasonFromError(...) could return no permanent-auth reason for this provider state.

Regression Test Plan

src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts: deactivated_workspace and deactivated workspace match permanent auth.
src/agents/failover-error.test.ts: the same markers resolve to auth_permanent through resolveFailoverReasonFromError(...).

User-visible / Behavior Changes

Provider errors containing deactivated_workspace are treated as a permanent auth/failover condition instead of an unknown classification.

Security Impact

New permissions/capabilities? No
Secrets/tokens handling changed? No
New/changed network calls? No
Command/tool execution surface changed? No
Data access scope changed? No

Real behavior proof

Behavior or issue addressed: OpenClaw cron/agent runs using openai-codex/gpt-5.4 and openai-codex/gpt-5.3-codex hit a real upstream Codex response {"detail":{"code":"deactivated_workspace"}}; before this patch, that marker was not in the permanent-auth matcher set.
Real environment tested: macOS local OpenClaw environment. Real pre-fix runtime evidence came from OpenClaw cron run logs for provider=openai-codex, model=gpt-5.4 / gpt-5.3-codex, redacted to omit run/session IDs. After-fix verification ran on this PR head 27c56e8192 from current upstream/main 4505a88d88.
Exact steps or command run after this patch:

node --import tsx -e "const { resolveFailoverReasonFromError, coerceToFailoverError } = await import('./src/agents/failover-error.ts'); const samples = ['deactivated_workspace', 'deactivated workspace', '{\"detail\":{\"code\":\"deactivated_workspace\"}}']; for (const message of samples) console.log(JSON.stringify({ message, reason: resolveFailoverReasonFromError({ message }), coercedReason: coerceToFailoverError({ message }, { provider: 'openai-codex', model: 'gpt-5.4' })?.reason }));"
./node_modules/.bin/vitest run src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts src/agents/failover-error.test.ts

Evidence after fix:

{"message":"deactivated_workspace","reason":"auth_permanent","coercedReason":"auth_permanent"}
{"message":"deactivated workspace","reason":"auth_permanent","coercedReason":"auth_permanent"}
{"message":"{\"detail\":{\"code\":\"deactivated_workspace\"}}","reason":"auth_permanent","coercedReason":"auth_permanent"}

Test Files  2 passed (2)
Tests       210 passed (210)

Redacted real pre-fix OpenClaw runtime samples:

cron run: status=error error={"detail":{"code":"deactivated_workspace"}} model=gpt-5.4 provider=openai-codex
cron run: status=error error={"detail":{"code":"deactivated_workspace"}} model=gpt-5.3-codex provider=openai-codex
embedded run: isError=true model=gpt-5.4 provider=openai-codex rawError={"detail":{"code":"deactivated_workspace"}}

Observed result after fix: The real Codex marker and the exact JSON payload shape from OpenClaw logs now normalize to auth_permanent through the same failover classification/coercion boundary used before model fallback handling.
What was not tested: I did not complete a live networked patched-gateway run against an externally deactivated Codex workspace. The local source CLI attempted a rebuild/dependency convergence step and hit npm registry optional-package download failures, so this proof combines real OpenClaw upstream error logs with after-fix execution of the patched failover classification path rather than a full live after-fix gateway E2E.

Verification

./node_modules/.bin/vitest run src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts src/agents/failover-error.test.ts
Result: 2 files passed, 210 tests passed.

Compatibility / Migration

Backward compatible? Yes
Config/env changes? No
Migration needed? No

Risks and Mitigations

Risk: deactivated_workspace is provider-specific and is mapped into the existing auth_permanent bucket.
Mitigation: the matcher is intentionally narrow (deactivated[_ ]workspace) and only changes classification for this marker.

Current-head maintainer proof after rebase

Current PR head tested: 3aa770fa848bf3a9701484d4aa23bcdbeb508a79
Base after rebase: origin/main 1d5b5db4d221b6d6b81d40c0c05cc644796d5b3b
Real environment tested: Crabbox Cloudflare Containers, provider cloudflare, lease cbx_5b879c7b4630.
Command shape: current checkout synced to Cloudflare, dependencies installed with pnpm install --frozen-lockfile, then node --import tsx imported src/agents/failover-error.ts and exercised the failover classification/coercion boundary.

Current-head output:

{"name":"message","reason":"auth_permanent","coercedReason":"auth_permanent"}
{"name":"detailCode","reason":"auth_permanent","coercedReason":"auth_permanent"}
{"name":"wrapped403DetailCode","reason":"auth_permanent","coercedReason":"auth_permanent"}
{"name":"controlMissingMarker","reason":null,"coercedReason":null}
cloudflare run summary sync=14.982s command=17.937s total=35.15s exit=0

Current-head CI/proof status:

GitHub Real behavior proof passed for the rebased PR head and reported: External PR includes after-fix real behavior proof.
GitHub PR checks reported success/skipped for the current head before this re-review request.

greptile-apps · 2026-03-27T16:14:09Z

Greptile Summary

This PR adds deactivated_workspace to the authPermanent error-pattern matcher in failover-matches.ts, so that provider errors containing this marker are correctly classified as a permanent auth/failover condition rather than falling through as an unknown error. Two targeted test cases are added — one at the isAuthPermanentErrorMessage helper level and one at the resolveFailoverReasonFromError / classifyFailoverReason level — to lock in the new behaviour.

src/agents/pi-embedded-helpers/failover-matches.ts: adds /deactivated[_ ]workspace/i regex to ERROR_PATTERNS.authPermanent.
src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts: adds "deactivated_workspace" to the isAuthPermanentErrorMessage positive samples and a direct classifyFailoverReason assertion.
src/agents/failover-error.test.ts: adds an it block asserting resolveFailoverReasonFromError({ message: "deactivated_workspace" }) returns "auth_permanent".

The change is narrowly scoped to error-string classification and does not touch fallback ordering, cooldown policy, auth storage, or any network behaviour.

Confidence Score: 5/5

Safe to merge — change is a single-line regex addition with full unit-test coverage and no side effects on other classification paths.

The fix is minimal, correctly placed, and well-tested. All existing patterns and tests are unaffected. No P0 or P1 issues found.

No files require special attention.

Important Files Changed

Filename	Overview
src/agents/pi-embedded-helpers/failover-matches.ts	Adds /deactivated[_ ]workspace/i regex to authPermanent pattern list — minimal, correct, and consistent with existing patterns.
src/agents/failover-error.test.ts	Adds a targeted test asserting deactivated_workspace resolves to auth_permanent at the failover-reason level.
src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts	Adds deactivated_workspace to the isAuthPermanentErrorMessage positive samples and a classifyFailoverReason assertion.

_{Reviews (1): Last reviewed commit: "Merge branch 'main' of ssh://ssh.github...." | Re-trigger Greptile}

litang9 · 2026-03-27T16:24:29Z

@codex review

chatgpt-codex-connector · 2026-03-27T16:27:13Z

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

clawsweeper · 2026-04-28T14:43:12Z

Codex review: needs maintainer review before merge.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR classifies Codex deactivated_workspace errors as permanent auth failures, reads nested detail.code, adds focused failover tests, and records the user-visible fix in the changelog.

Reproducibility: yes. source-reproducible: on current main, a { detail: { code: "deactivated_workspace" } } Codex payload is not read as an error code, and the marker is absent from the permanent-auth matcher. I did not execute tests because this review was read-only.

PR rating
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Summary: The PR is small, well-scoped, covered at the right classifier boundaries, and now has current-head real behavior proof.

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Sufficient (terminal): The PR body includes current-head terminal proof for 3aa770fa848bf3a9701484d4aa23bcdbeb508a79 plus redacted real pre-fix Codex runtime samples, and the observed after-fix output shows the classifier/coercion boundary returning auth_permanent.

Next step before merge
No repair lane is needed because the PR already contains the narrow code/test fix and sufficient current-head proof; the remaining action is ordinary maintainer review or merge.

Security
Cleared: The diff only changes failover classification, tests, and changelog text; it does not alter dependencies, CI, permissions, secrets, network calls, or code execution paths.

Review details

Best possible solution:

Land this narrow classifier mitigation with the refreshed current-head proof, while leaving active workspace selection and recovery behavior to #56693.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: on current main, a { detail: { code: "deactivated_workspace" } } Codex payload is not read as an error code, and the marker is absent from the permanent-auth matcher. I did not execute tests because this review was read-only.

Is this the best way to solve the issue?

Yes for the mitigation: adding a narrow marker plus nested detail.code extraction at the failover boundary is the maintainable fix for this symptom. The broader account/workspace recovery behavior should stay in the linked follow-up issue.

Label changes:

add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes current-head terminal proof for 3aa770fa848bf3a9701484d4aa23bcdbeb508a79 plus redacted real pre-fix Codex runtime samples, and the observed after-fix output shows the classifier/coercion boundary returning auth_permanent.
add rating: 🦞 diamond lobster: Current PR rating is 🦞 diamond lobster because proof is 🦞 diamond lobster, patch quality is 🦞 diamond lobster, and The PR is small, well-scoped, covered at the right classifier boundaries, and now has current-head real behavior proof.
add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes current-head terminal proof for 3aa770fa848bf3a9701484d4aa23bcdbeb508a79 plus redacted real pre-fix Codex runtime samples, and the observed after-fix output shows the classifier/coercion boundary returning auth_permanent.
remove rating: 🦐 gold shrimp: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.
remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

P2: This is a normal-priority agent failover bug fix with limited blast radius and focused tests.
rating: 🦞 diamond lobster: Current PR rating is 🦞 diamond lobster because proof is 🦞 diamond lobster, patch quality is 🦞 diamond lobster, and The PR is small, well-scoped, covered at the right classifier boundaries, and now has current-head real behavior proof.
status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The PR body includes current-head terminal proof for 3aa770fa848bf3a9701484d4aa23bcdbeb508a79 plus redacted real pre-fix Codex runtime samples, and the observed after-fix output shows the classifier/coercion boundary returning auth_permanent.
proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes current-head terminal proof for 3aa770fa848bf3a9701484d4aa23bcdbeb508a79 plus redacted real pre-fix Codex runtime samples, and the observed after-fix output shows the classifier/coercion boundary returning auth_permanent.

What I checked:

Current main lacks the Codex marker: Current main's high-confidence permanent-auth matcher includes revoked/deactivated API keys, disabled/revoked keys, deactivated accounts, and organization blocks, but not deactivated_workspace. (src/agents/pi-embedded-helpers/failover-matches.ts:8, 1d5b5db4d221)
Current main does not read nested detail.code: Current main reads direct code or nonnumeric string status as an error code, so a Codex payload shaped like { detail: { code: "deactivated_workspace" } } is not surfaced to the classifier. (src/agents/failover-error.ts:145, 1d5b5db4d221)
Model fallback contract covers auth failures: The docs state that when all profiles for a provider fail, OpenClaw moves to configured fallbacks for auth failures, rate limits, and timeouts that exhaust profile rotation. Public docs: docs/concepts/model-failover.md. (docs/concepts/model-failover.md:258, 1d5b5db4d221)
PR reads nested Codex error codes: The PR head adds detail.code extraction alongside direct code, making the observed Codex JSON error shape available to failover classification. (src/agents/failover-error.ts:154, 3aa770fa848b)
PR maps deactivated workspace to permanent auth: The PR head maps DEACTIVATED_WORKSPACE to auth_permanent and evaluates that permanent-auth code before HTTP status classification can collapse 400 payloads into generic format errors. (src/agents/pi-embedded-helpers/errors.ts:761, 3aa770fa848b)
PR adds focused regression coverage: The PR head tests message, direct code, nested detail.code, and 400/403 wrapper shapes resolving to auth_permanent. (src/agents/failover-error.test.ts:946, 3aa770fa848b)

Likely related people:

steipete: Peter Steinberger introduced shared failover error matchers and later unified structured provider failover classification in the central files this PR touches. (role: feature-history contributor; confidence: high; commits: 6472e03412ef, cd4b03c5684f, 131f6dac37a8; files: src/agents/failover-error.ts, src/agents/pi-embedded-helpers/errors.ts, src/agents/pi-embedded-helpers/failover-matches.ts)
altaywtf: Altay previously shared failover HTTP status classification, added fallback error observations, and contributed adjacent failover classifier fixes in the same agent surface. (role: recent area contributor; confidence: high; commits: f014e255dfb0, 531e8362b1bd, 49acb07f9f0a; files: src/agents/failover-error.ts, src/agents/pi-embedded-helpers/errors.ts, src/agents/model-fallback.ts)
Eva: Eva recently touched OpenAI Codex auth/runtime classification and the Codex OAuth provider path that are adjacent to this provider-specific classifier mitigation. (role: adjacent Codex auth contributor; confidence: medium; commits: 8166d592d92f; files: src/agents/pi-embedded-helpers/errors.ts, src/agents/pi-embedded-helpers.isbillingerrormessage.test.ts, src/plugins/provider-openai-codex-oauth.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 1d5b5db4d221.

clawsweeper · 2026-05-20T03:38:48Z

ClawSweeper PR egg

✨ Hatched: 🌱 uncommon Moonlit Review Wisp

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

Merged PRs are hatchable.
Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.

Rarity: 🌱 uncommon.
Trait: finds missing screenshots.
Image traits: location diff observatory; accessory commit compass; palette rose quartz and slate; mood bright-eyed; pose stepping out of a freshly hatched shell; shell polished stone shell; lighting gentle morning glow; background little resolved-comment flags.
Share on X: post this hatch
Copy: My PR egg hatched a 🌱 uncommon Moonlit Review Wisp in ClawSweeper.

What is this egg doing here?

Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

altaywtf · 2026-05-21T19:27:11Z

@clawsweeper re-review

clawsweeper · 2026-05-21T19:29:27Z

🦞🧹
ClawSweeper could not start a re-review for this item.

Reason: re-review requires an open issue or PR.

altaywtf · 2026-05-21T19:31:09Z

@clawsweeper re-review

altaywtf · 2026-05-21T19:47:37Z

Merged via squash.

Prepared head SHA: 3aa770fa848bf3a9701484d4aa23bcdbeb508a79
Merge commit: 016c34ff1d2ac4cb7521c8cfe4240581332bef09

Thanks @litang9!

@altaywtf