Skip to content

fix(gateway): canonicalize fallback model override keys#85445

Closed
nxmxbbd wants to merge 1 commit into
openclaw:mainfrom
nxmxbbd:nex/gateway-fallback-modelkey-20260522222435
Closed

fix(gateway): canonicalize fallback model override keys#85445
nxmxbbd wants to merge 1 commit into
openclaw:mainfrom
nxmxbbd:nex/gateway-fallback-modelkey-20260522222435

Conversation

@nxmxbbd

@nxmxbbd nxmxbbd commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: gateway fallback subagent model-override allowlist checks still raw-concatenated normalized provider/model refs, so provider-prefixed OpenRouter requests could report openrouter/openrouter/... in deny diagnostics.
  • Solution: use the existing canonical modelKey() helper for both the configured allowlist side and requested fallback override side.
  • What changed: src/gateway/server-plugins.ts now canonicalizes the three fallback comparison keys, and src/gateway/server-plugins.test.ts covers the OpenRouter deny diagnostic plus matching allow path.
  • What did NOT change (scope boundary): no plugin config schema changes, no runtime LLM changes, no provider dispatch payload changes, no auth/SecretRef behavior changes, and no docs/changelog changes.

Motivation

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: gateway fallback subagent model-override allowlist diagnostics now use canonical provider/model refs for provider-prefixed OpenRouter requests, and the matching allow path reaches the gateway agent handler instead of being falsely denied by fallback policy key comparison.
  • Real environment tested: local Linux source checkout on Node 22.22.1, branch rebased on last-known-green upstream/main faf2a6cb9e (CI run 26298182034 succeeded) with head f4ee9b72ee. I ran a production-source gateway fallback smoke invocation through createGatewaySubagentRuntime() plus the real gateway agent handler validation path, and also ran Vitest 4.1.7 via node scripts/run-vitest.mjs.
  • Exact steps or command run after this patch:
node --import tsx <production-source-smoke-script> .
node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway.config.ts src/gateway/server-plugins.test.ts -t 'OpenRouter fallback'
node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway.config.ts src/gateway/server-plugins.test.ts

The production-source smoke script imports src/gateway/server-plugins.ts and src/plugins/runtime/gateway-request-scope.ts, configures plugin voice-call with subagent.allowModelOverride: true and subagent.allowedModels: ["openrouter/gpt-5.4-mini"], then calls createGatewaySubagentRuntime().run() under plugin scope. The matching-allow case intentionally uses a malformed session key so the invocation stops in the real gateway agent handler validation before any agent/provider execution.

  • Evidence after fix:

Terminal capture from this branch, copied live output with local paths redacted:

{
  "repo": "<local source checkout>",
  "proofKind": "production-source gateway fallback subagent invocation; no provider/network call",
  "policy": {
    "allowModelOverride": true,
    "allowedModels": [
      "openrouter/gpt-5.4-mini"
    ]
  },
  "deny": {
    "name": "deny disallowed provider-prefixed OpenRouter model",
    "ok": false,
    "error": "model override \"openrouter/gpt-5.5\" is not allowlisted for plugin \"voice-call\"."
  },
  "allow": {
    "name": "allow matching provider-prefixed OpenRouter model",
    "ok": false,
    "error": "invalid agent params: malformed session key \"agent::allow-proof\""
  },
  "calls": {
    "addChatRun": 0,
    "removeChatRun": 0,
    "broadcast": 0,
    "registerToolEventRecipient": 0
  },
  "dedupeEntries": 0,
  "chatAbortControllers": 0,
  "checks": {
    "denyCanonical": true,
    "denyNoDuplicateProvider": true,
    "allowReachedGatewayValidation": true,
    "noAgentRunStarted": true
  }
}

Supplemental regression-test output from the same branch:

$ node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway.config.ts src/gateway/server-plugins.test.ts -t 'OpenRouter fallback'
RUN v4.1.7
✓ gateway src/gateway/server-plugins.test.ts (47 tests | 45 skipped) 4190ms

Test Files 1 passed (1)
Tests 2 passed | 45 skipped (47)
Duration 4.43s

$ node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway.config.ts src/gateway/server-plugins.test.ts
RUN v4.1.7
✓ gateway src/gateway/server-plugins.test.ts (47 tests) 4132ms

Test Files 1 passed (1)
Tests 47 passed (47)
Duration 4.42s
  • Observed result after fix: the production-source smoke invocation returns the canonical deny diagnostic model override "openrouter/gpt-5.5" without openrouter/openrouter/. The matching allowlisted request reaches real gateway agent handler validation (malformed session key "agent::allow-proof") instead of failing fallback override authorization, while counters confirm no agent/provider run started. The focused OpenRouter fallback tests and the full server-plugins.test.ts file also pass.
  • What was not tested: no live provider/network call and no actual agent run were made; the production-source smoke intentionally stops at gateway validation after fallback policy authorization. Full repo CI is not claimed. Exact-latest upstream main CI is not claimed green during local prep; this branch is based on the last-known-green main SHA faf2a6cb9e, while newer main runs were pending/in-progress or had failing jobs when checked.
  • Before evidence (optional but encouraged): the same deny test failed on current main before the code change with model override "openrouter/openrouter/gpt-5.5" is not allowlisted for plugin "voice-call".

Root Cause (if applicable)

  • Root cause: normalizeModelRef() can return { provider: "openrouter", model: "openrouter/..." } when the request model already carries the provider prefix. The gateway fallback policy then formatted keys with raw `${provider}/${model}` instead of the canonical helper that dedupes an already provider-prefixed model id.
  • Missing detection / guardrail: gateway fallback tests covered generic trusted and untrusted override behavior, but did not assert the provider-prefixed OpenRouter diagnostic shape or pair that deny test with a matching allow-path guard.
  • Contributing context (if known): fix #84887: avoid duplicate provider prefix in runtime LLM allowlist diagnostics #84946 intentionally fixed the runtime LLM policy path only; this PR handles the remaining gateway fallback policy path separately.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: src/gateway/server-plugins.test.ts
  • Scenario the test should lock in: a trusted fallback subagent with allowedModels: ["openrouter/gpt-5.4-mini"] rejects provider: "openrouter", model: "openrouter/gpt-5.5" without a duplicate provider prefix, and still allows the matching provider: "openrouter", model: "openrouter/gpt-5.4-mini" request.
  • Why this is the smallest reliable guardrail: it exercises the same fallback policy builder, allowlist comparison, rejection diagnostic, and dispatch payload path without requiring a live provider call.
  • Existing test that already covers this (if any): existing fallback override tests covered generic non-OpenRouter shapes and model-only canonical refs, but not provider-prefixed OpenRouter refs.
  • If no new test is added, why not: N/A; new regression tests are added.

User-visible / Behavior Changes

Gateway fallback subagent override denial messages now report the canonical OpenRouter model ref instead of a duplicate provider-prefixed ref. Matching allowlisted provider-prefixed OpenRouter requests remain authorized and dispatch unchanged.

Diagram (if applicable)

N/A

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: Linux
  • Runtime/container: Node 22.22.1 in a local source checkout
  • Model/provider: OpenRouter model-ref strings only; no live model/provider call
  • Integration/channel (if any): gateway fallback subagent runtime test harness
  • Relevant config (redacted): plugin voice-call with subagent.allowModelOverride: true and subagent.allowedModels: ["openrouter/gpt-5.4-mini"]

Steps

  1. Configure a trusted plugin fallback subagent allowlist with allowedModels: ["openrouter/gpt-5.4-mini"].
  2. Deny-path check: request provider: "openrouter", model: "openrouter/gpt-5.5".
  3. Allow-path check: request provider: "openrouter", model: "openrouter/gpt-5.4-mini".
  4. Run node scripts/run-vitest.mjs run --config test/vitest/vitest.gateway.config.ts src/gateway/server-plugins.test.ts -t 'OpenRouter fallback'.

Expected

  • Deny-path diagnostic uses openrouter/gpt-5.5 and does not contain openrouter/openrouter/.
  • Allow-path request dispatches with provider: "openrouter" and model: "openrouter/gpt-5.4-mini".

Actual

  • Before this patch, the deny-path diagnostic contained openrouter/openrouter/gpt-5.5.
  • After this patch, the focused OpenRouter fallback tests and the full server-plugins.test.ts file pass.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

  • Verified scenarios: focused OpenRouter deny diagnostic; matching OpenRouter allow path; full src/gateway/server-plugins.test.ts; diff review confirmed only src/gateway/server-plugins.ts and src/gateway/server-plugins.test.ts changed.
  • Edge cases checked: configured allowlist side and requested override side both use the same canonical helper; dispatch params remain caller-supplied provider/model values, not rewritten payload values.
  • What you did not verify: live provider/network behavior and full repo CI.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

No review conversations exist yet for this PR.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: canonicalizing only one side of the comparison could create false denies for matching allowlisted overrides.
    • Mitigation: the patch canonicalizes both allowlist and requested override keys, and adds a matching allow-path regression test.
  • Risk: diagnostic-only tests could miss over-correction that changes the dispatch payload.
    • Mitigation: the allow-path test asserts the dispatched params still preserve provider: "openrouter" and model: "openrouter/gpt-5.4-mini".

@openclaw-barnacle openclaw-barnacle Bot added gateway Gateway runtime size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Thanks for the context here. I swept through the related work, and this is now duplicate or superseded.

Keep open for normal maintainer review: the fallback-policy bug is source-reproducible on current main, the PR is focused, the supplied live-output proof applies to the current head, and I found no blocking correctness or security findings.

Canonical path: Close this PR as superseded by #84946.

So I’m closing this here and keeping the remaining discussion on #84946.

Review details

Best possible solution:

Close this PR as superseded by #84946.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows the fallback allowlist and requested override paths still raw-concatenate normalized provider/model refs, and the PR body supplies before/after output for the duplicate OpenRouter prefix and matching allow path.

Is this the best way to solve the issue?

Yes. Reusing the existing canonical modelKey() helper on both configured and requested comparison keys is the narrowest maintainable fix and matches the already-merged runtime LLM policy solution.

Security review:

Security review cleared: The diff only changes host-side model-ref key canonicalization and focused tests, with no dependency, workflow, secret, permission, network, or code-execution surface added.

What I checked:

Likely related people:

  • OmerZeyveli: Bounded git blame on current main attributes the gateway fallback allowlist formatter and the shared modelKey() helper to commit 14b2b8ac... by author display name Riive, whose noreply metadata maps to this GitHub login. (role: introduced behavior in available history; confidence: medium; commits: 14b2b8ac4824; files: src/gateway/server-plugins.ts, src/gateway/server-plugins.test.ts, src/agents/model-ref-shared.ts)
  • steipete: Recent current-main history includes commit 658be7f1... touching the gateway files, and the current PR head b0932cf... is committed by Peter Steinberger after maintainer force-pushes on the PR branch. (role: recent area contributor and PR committer; confidence: medium; commits: 658be7f1c721, b0932cf832af; files: src/gateway/server-plugins.ts, src/gateway/server-plugins.test.ts)
  • zhangguiping-xydt: The merged related runtime PR at fix #84887: avoid duplicate provider prefix in runtime LLM allowlist diagnostics #84946 applied the same modelKey() solution to the runtime LLM allowlist path, which is the direct precedent for this gateway fallback change. (role: adjacent merged fix author; confidence: medium; commits: 937a756f7ffc; files: src/plugins/runtime/runtime-llm.runtime.ts, src/plugins/runtime/runtime-llm.runtime.test.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 1cd6dce07550.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal backlog priority with limited blast radius. labels May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

✨ Hatched: 🥚 common Mossy Patch Peep

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.

Rarity: 🥚 common.
Trait: polishes edge cases.
Image traits: location branch lighthouse; accessory lint brush; palette moss green and polished brass; mood celebratory; pose sitting proudly on a smooth stone; shell glossy opal shell; lighting gentle morning glow; background subtle branch markers.
Share on X: post this hatch
Copy: My PR egg hatched a 🥚 common Mossy Patch Peep in ClawSweeper.

What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 22, 2026
@nxmxbbd nxmxbbd force-pushed the nex/gateway-fallback-modelkey-20260522222435 branch from f4ee9b7 to 0be4971 Compare May 22, 2026 18:50
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@steipete steipete force-pushed the nex/gateway-fallback-modelkey-20260522222435 branch from 0be4971 to 805588a Compare May 22, 2026 22:28
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@steipete steipete force-pushed the nex/gateway-fallback-modelkey-20260522222435 branch from 805588a to b0932cf Compare May 22, 2026 22:48
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper applied the proposed close for this PR.

@clawsweeper clawsweeper Bot closed this May 22, 2026
@nxmxbbd

nxmxbbd commented May 23, 2026

Copy link
Copy Markdown
Contributor Author

@steipete The close evidence treats #84946 as superseding this PR, but #84946 does not contain the gateway-side fix and current main still lacks it.

Evidence:

  • fix #84887: avoid duplicate provider prefix in runtime LLM allowlist diagnostics #84946 is merged, but it only touched src/plugins/runtime/runtime-llm.runtime.ts and src/plugins/runtime/runtime-llm.runtime.test.ts.
  • This PR touches src/gateway/server-plugins.ts and src/gateway/server-plugins.test.ts.
  • Current main (7f05be0) still raw-concatenates fallback model keys in src/gateway/server-plugins.ts: the configured allowlist path and both requested override paths still return ${provider}/${model}-style strings instead of modelKey().
  • fix: model ref normalization, Vertex transport routing, symlink workspace dirs #84934 overlaps only partially: it normalizes the requested-side gateway key, but does not change the configured allowlist normalization path.
  • The patch currently on this PR's head (b0932cf) uses modelKey() for both configured and requested gateway fallback comparison keys, and the patch content is unchanged from the branch you rebased.

The durable review linked as close evidence also says current main still raw-concatenates these gateway fallback refs and that using modelKey() on configured + requested comparison keys is the narrowest fix. That makes #84946 a related runtime precedent, not a superseding gateway implementation.

Could this PR be reopened, or is there another PR/commit that already landed the full gateway-side fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gateway Gateway runtime P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant