Skip to content

fix: CLI auth status/probe path regression#75223

Open
clawsweeper[bot] wants to merge 4 commits into
mainfrom
clawsweeper/clawsweeper-commit-openclaw-openclaw-581fbea1d653
Open

fix: CLI auth status/probe path regression#75223
clawsweeper[bot] wants to merge 4 commits into
mainfrom
clawsweeper/clawsweeper-commit-openclaw-openclaw-581fbea1d653

Conversation

@clawsweeper

@clawsweeper clawsweeper Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Found one regression in the CLI auth status/probe path. The commit intentionally makes unscoped external CLI discovery skip missing runtime-only profiles, but models status and models status --probe still load the auth store without passing the providers they are about to report or probe.

What ClawSweeper Is Fixing

  • Medium: models status --probe misses CLI-only credentials after scoped discovery (regression)
    • File: src/commands/models/list.probe.ts:259
    • Evidence: src/agents/auth-profiles/external-cli-sync.ts:210 now returns false for unscoped missing external CLI profiles unless a matching stored OAuth profile already exists. buildProbeTargets still calls ensureAuthProfileStore(agentDir) without allowKeychainPrompt: false, externalCliProviderIds, or externalCliProfileIds, even though the function receives the exact providers and probe profile filters at src/commands/models/list.probe.ts:254-256. A focused smoke with a temporary Codex CLI auth file confirmed resolveExternalCliAuthProfiles(emptyStore) returns [], while resolveExternalCliAuthProfiles(emptyStore, { providerIds: ["openai-codex"] }) returns ["openai-codex:default"].
    • Impact: A user who only has file-backed Codex/Claude/Minimax CLI credentials and no persisted OpenClaw auth profile can resolve runtime auth through the newly scoped model-auth path, but openclaw models status --probe will build no profile target for that same credential. This produces false “nothing to probe” or missing-auth diagnostics for the provider most likely being checked.
    • Suggested fix: In buildProbeTargets, load the store with a scope derived from the requested probe surface, for example ensureAuthProfileStore(agentDir, { allowKeychainPrompt: false, config: cfg, externalCliProviderIds: providers, externalCliProfileIds: options.profileIds }). The plain status path at src/commands/models/list.status-command.ts:223 should likewise pass the config-derived external CLI scope, matching src/gateway/server-methods/models-auth-status.ts:296-301.
    • Confidence: high

Expected Repair Surface

  • src/commands/models/list.probe.ts
  • src/agents/auth-profiles.external-cli-scope.test.ts
  • src/agents/auth-profiles.external-cli-sync.test.ts
  • src/agents/auth-profiles/external-cli-scope.ts
  • src/agents/auth-profiles/external-cli-sync.ts
  • src/agents/model-auth.profiles.test.ts
  • src/agents/model-auth.ts

Source And Review Context

  • ClawSweeper report: https://github.com/openclaw/clawsweeper/blob/main/records/openclaw-openclaw/commits/581fbea1d65355917886babffe4161f8b3178a12.md

  • Commit under review: 581fbea

  • Latest main at intake: 581fbea

  • Original commit author: Peter Steinberger

  • GitHub author: @steipete

  • Highest severity: medium

  • Review confidence: high

  • Diff: 54e6e3d7daf5d0d857edf756b35628a29d11c7f5..581fbea1d65355917886babffe4161f8b3178a12

  • Changed files: src/agents/auth-profiles.external-cli-scope.test.ts, src/agents/auth-profiles.external-cli-sync.test.ts, src/agents/auth-profiles/external-cli-scope.ts, src/agents/auth-profiles/external-cli-sync.ts, src/agents/model-auth.profiles.test.ts, src/agents/model-auth.ts

  • Code read: changed files, auth store overlay path, external auth overlay, auth profile order, CLI model status/probe callers, gateway auth-status caller

  • Dependencies/web: no dependency changes; no web lookup needed

Expected validation

  • pnpm check:changed

ClawSweeper already ran:

  • pnpm docs:list
  • pnpm install --frozen-lockfile
  • Focused external CLI resolver smoke with temporary Codex CLI auth file
  • pnpm test src/agents/auth-profiles.external-cli-sync.test.ts src/agents/auth-profiles.external-cli-scope.test.ts src/agents/model-auth.profiles.test.ts src/commands/models/list.probe.targets.test.ts
  • git diff --check 54e6e3d7daf5d0d857edf756b35628a29d11c7f5..581fbea1d65355917886babffe4161f8b3178a12

Known review limits:

  • I did not run a full models status --probe live provider call, because the actionable regression is in target construction before network probing.

ClawSweeper Guardrails

  • Re-check the finding against latest main before changing code.
  • Keep the patch to the narrowest behavior change and matching regression coverage.
  • Do not merge automatically; this PR stays for maintainer review.

ClawSweeper 🐠 replacement reef notes:

  • Cluster: clawsweeper-commit-openclaw-openclaw-581fbea1d653
  • Source PRs: none
  • Credit: Detected by ClawSweeper commit review for 581fbea.; Original commit author: Peter Steinberger.
  • Validation: pnpm check:changed

fish notes: model gpt-5.5, reasoning medium; reviewed against 24edcd5.

@clawsweeper clawsweeper Bot added clawsweeper Tracked by ClawSweeper automation clawsweeper:commit-finding PR created from a ClawSweeper commit finding labels Apr 30, 2026
@openclaw-barnacle openclaw-barnacle Bot added commands Command implementations size: S labels Apr 30, 2026
@clawsweeper

clawsweeper Bot commented Apr 30, 2026

Copy link
Copy Markdown
Contributor Author

Codex review: needs changes before merge. Reviewed May 31, 2026, 1:01 AM ET / 05:01 UTC.

Summary
The branch scopes external CLI auth discovery for CLI model status/probe paths, adds regression tests, and also edits the changelog plus one unrelated cron test type.

PR surface: Source +6, Tests +112, Docs +1. Total +119 across 6 files.

Reproducibility: yes. source-reproducible but not live-run here: current main's plain models status path loads the no-external-profile store while external CLI discovery requires a provider/profile scope to surface missing runtime-only credentials.

Review metrics: 1 noteworthy metric.

  • Auth discovery entry points: 2 CLI paths changed. The PR changes both status reporting and probe target construction, so maintainers should review the auth-provider behavior as one status/probe surface.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🦐 gold shrimp
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh or replace the branch on current main.
  • Drop CHANGELOG.md and unrelated cron test changes.
  • Run focused status/probe tests after the refresh.

Risk before merge

  • [P1] The branch is dirty against current main, so maintainers should review a refreshed merge result rather than the stale patch shape.
  • [P1] The diff changes auth-provider discovery for CLI status/probe; focused tests are useful, but a refreshed branch should prove the CLI-only status path on current main and preserve allowKeychainPrompt: false.
  • [P1] The branch edits CHANGELOG.md and an unrelated cron test, which should be dropped from a normal narrow bug-fix PR.

Maintainer options:

  1. Refresh and port only the auth fix (recommended)
    Rebase or replace the branch on current main, keep the scoped external CLI auth behavior for status/probe, drop the changelog and unrelated cron changes, then rerun focused status/probe tests.
  2. Pause for broader auth review
    If maintainers want to revisit the auth-status discovery contract more broadly, hold this branch and make the desired status/probe behavior explicit before repair.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Refresh this PR onto current main, keep only the CLI models status/probe external CLI auth discovery fix plus focused tests, remove CHANGELOG.md and unrelated src/cron/isolated-agent.model-formatting.test.ts changes, then run node scripts/run-vitest.mjs src/commands/models/list.probe.targets.test.ts src/commands/models/list.status.test.ts and git diff --check.

Next step before merge

  • [P2] A repair worker can refresh the branch and make the narrow status/probe fix mergeable without needing a product decision.

Security
Cleared: No concrete security or supply-chain regression found; the diff touches auth status/probe discovery but does not add dependencies, CI execution, or visible credential output.

Review findings

  • [P3] Remove the release-owned changelog edit — CHANGELOG.md:16
Review details

Best possible solution:

Port the narrow external-CLI status/probe discovery fix onto current main with focused regression coverage and no changelog or unrelated test churn.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible but not live-run here: current main's plain models status path loads the no-external-profile store while external CLI discovery requires a provider/profile scope to surface missing runtime-only credentials.

Is this the best way to solve the issue?

Unclear as submitted: the status-store change is the right narrow direction, but the branch needs to be refreshed onto current main and trimmed before it is the best mergeable fix.

Full review comments:

  • [P3] Remove the release-owned changelog edit — CHANGELOG.md:16
    CHANGELOG.md is release-owned for normal PRs in this repo. Please move this release-note context to the PR body or eventual squash message instead of landing a source changelog edit here.
    Confidence: 0.9

Overall correctness: patch is correct
Overall confidence: 0.78

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against e1a98171417c.

Label changes

Label changes:

  • add P2: This is a normal-priority CLI auth-status/probe regression with limited blast radius but real operator-facing false-missing-auth impact.
  • add merge-risk: 🚨 auth-provider: The patch changes how CLI status/probe discovers external auth profiles, which can affect provider auth reporting and probe target selection.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🦐 gold shrimp.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a bot-authored ClawSweeper PR, so the external-contributor real-behavior proof gate does not apply; the body reports targeted tests and a temporary CLI auth resolver smoke, not a live provider probe.
  • remove rating: 🌊 off-meta tidepool: Current PR rating is rating: 🦐 gold shrimp, so this older rating label is no longer current.

Label justifications:

  • P2: This is a normal-priority CLI auth-status/probe regression with limited blast radius but real operator-facing false-missing-auth impact.
  • merge-risk: 🚨 auth-provider: The patch changes how CLI status/probe discovers external auth profiles, which can affect provider auth reporting and probe target selection.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🦐 gold shrimp.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: This is a bot-authored ClawSweeper PR, so the external-contributor real-behavior proof gate does not apply; the body reports targeted tests and a temporary CLI auth resolver smoke, not a live provider probe.
Evidence reviewed

PR surface:

Source +6, Tests +112, Docs +1. Total +119 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 2 11 5 +6
Tests 3 125 13 +112
Docs 1 1 0 +1
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 137 18 +119

Acceptance criteria:

  • [P1] node scripts/run-vitest.mjs src/commands/models/list.probe.targets.test.ts src/commands/models/list.status.test.ts.
  • [P1] git diff --check.
  • [P1] node scripts/crabbox-wrapper.mjs run --shell -- "pnpm check:changed" if maintainers require the broad changed gate.

What I checked:

  • Current main status path still misses external CLI overlay: modelsStatusCommand imports ensureAuthProfileStoreWithoutExternalProfiles as ensureAuthProfileStore and calls it without an external CLI discovery scope, so CLI-only runtime credentials can still be absent from plain status output. (src/commands/models/list.status-command.ts:326, e1a98171417c)
  • Current main probe path is already partly fixed: buildProbeTargets already calls ensureAuthProfileStore with externalCliDiscoveryScoped, providerIds, profileIds, and allowKeychainPrompt: false, so the probe half is mostly present on current main. (src/commands/models/list.probe.ts:297, e1a98171417c)
  • External CLI discovery is scope-sensitive: When no provider/profile scope is supplied, external CLI discovery only considers matching existing OAuth profiles; scoped provider/profile IDs are what let missing runtime-only CLI profiles be discovered. (src/agents/auth-profiles/external-cli-sync.ts:264, e1a98171417c)
  • Proposed status fix direction: The PR branch changes modelsStatusCommand to load ensureAuthProfileStore with externalCliDiscoveryForConfigStatus({ cfg: resolvedConfig }), which matches the missing current-main status behavior. (src/commands/models/list.status-command.ts:241, 10312a90647e)
  • Branch needs cleanup before merge: The PR diff includes a normal-PR CHANGELOG.md edit and an unrelated cron test type-only change, both outside the narrow CLI auth status/probe repair. (CHANGELOG.md:16, 10312a90647e)
  • Branch is not directly mergeable: Provided GitHub context reports mergeable=false / mergeableState=dirty; local git diff main...10312a90647ecc0a1db39f5d9700c680f67bc8fc also failed with no merge base, so review should be against a refreshed branch.

Likely related people:

  • Peter Steinberger: Commit 581fbea1d65355917886babffe4161f8b3178a12 introduced scoped external CLI credential discovery, and current-main blame/log evidence for the status/probe area also points through recent release/current-main work by Peter. (role: introduced related behavior and recent area contributor; confidence: high; commits: 581fbea1d653, 40181afdedb0, 22e4289d3f05; files: src/agents/auth-profiles/external-cli-scope.ts, src/agents/auth-profiles/external-cli-sync.ts, src/agents/model-auth.ts)
  • Josh Avant: git log -G on the probe/status/auth scope paths shows adjacent credential-semantics work in commit 1c200ca7ae3cd4a3e2861b1a32fc16b917630f09. (role: adjacent credential-semantics contributor; confidence: medium; commits: 1c200ca7ae3c; files: src/commands/models/list.probe.ts, src/commands/models/list.status-command.ts, src/agents/auth-profiles/external-cli-scope.ts)
  • Jeremiah Lowin: git log -G 'ensureAuthProfileStore\(agentDir' shows dd4715a2c409cb70a47b0658a0ebe735ffb2ee99 adding the --agent flag to models status, an adjacent CLI status path. (role: adjacent CLI status contributor; confidence: medium; commits: dd4715a2c409; files: src/commands/models/list.status-command.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper

clawsweeper Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor Author

🦞🔧
ClawSweeper picked up the repair feedback.

Source: clawsweeper[bot]
Feedback: structured ClawSweeper marker: fix-required (finding=review-feedback sha=24edcd5731df51f73120fd219209853d59a06d3a)
Action: repair worker queued. Run: https://github.com/openclaw/clawsweeper/actions/runs/25937750377
Model: gpt-5.5

I will update this PR branch, or open a safe credited replacement, if the repair worker finds a narrow fix.

Automerge progress:

  • 2026-05-15 22:03:13 UTC review requested repair 24edcd5731df (structured ClawSweeper marker: fix-required (finding=review-feedback sha=24edcd...)

@clawsweeper clawsweeper Bot force-pushed the clawsweeper/clawsweeper-commit-openclaw-openclaw-581fbea1d653 branch from 24edcd5 to b2695bb Compare May 15, 2026 22:56
@barnacle-openclaw

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@barnacle-openclaw barnacle-openclaw Bot added the stale Marked as stale due to inactivity label May 30, 2026
@clawsweeper clawsweeper Bot added the rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. label May 30, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the stale Marked as stale due to inactivity label May 31, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. and removed rating: 🌊 off-meta tidepool PR readiness rating does not apply to this item. labels May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clawsweeper:commit-finding PR created from a ClawSweeper commit finding clawsweeper Tracked by ClawSweeper automation commands Command implementations merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants