Skip to content

fix(delegate-task): honor user fallback_models when category primary is unreachable#3982

Merged
code-yeongyu merged 1 commit into
code-yeongyu:devfrom
jas32096:fix/category-fallback-ignored-when-primary-set
May 15, 2026
Merged

fix(delegate-task): honor user fallback_models when category primary is unreachable#3982
code-yeongyu merged 1 commit into
code-yeongyu:devfrom
jas32096:fix/category-fallback-ignored-when-primary-set

Conversation

@jas32096

@jas32096 jas32096 commented May 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes a silent-failure path in resolveModelForDelegateTask where a user-configured categories.*.model that is not reachable on the user's connected providers was returned verbatim, bypassing the user's own fallback_models array. The team-mode category member (e.g. hyperplan's artistry) then spawned targeting an unreachable model.

The bug

src/tools/delegate-task/model-selection.ts, lines 57-64 (pre-patch):

const userModel = normalizeModel(input.userModel)
if (userModel) {
  const parsed = parseUserFallbackModel(userModel)
  if (parsed?.variant) {
    return { model: parsed.baseModel, variant: parsed.variant }
  }
  return { model: userModel }
}

The moment input.userModel was set, the function returned it immediately — never checking reachability against input.availableModels, never iterating input.userFallbackModels. For a user with:

"categories": {
  "artistry": {
    "model": "opencode/gemini-3.1-pro",
    "fallback_models": [
      { "model": "amazon-bedrock/us.anthropic.claude-opus-4-7", "variant": "max" },
      { "model": "openai/gpt-5.5" }
    ]
  }
}

… and a provider set that includes openai + amazon-bedrock but not opencode-hosted gemini, the team member was launched against opencode/gemini-3.1-pro and the user's carefully-authored fallbacks were silently ignored. Runtime-fallback can't intercept this either because the provider rejects the model up-front.

The fix

When the provider-models cache is warm (availableModels.size > 0) AND the user provided fallback_models, we now:

  1. Verify the user's primary model is reachable via fuzzyMatchModel. If yes → return it (fast path, identical to old behavior).
  2. If not reachable, iterate userFallbackModels and promote the first reachable entry, with matchedFallback: true.
  3. If neither reachable, return userResult as-is — preserving the legacy "trust the user" behavior.

Cold-cache first-run behavior is untouched (size > 0 guard). The existing pre-cache test at model-selection.test.ts:42-55 still passes unchanged — deliberate: on first run we have no authoritative availability data and must honor user intent.

Tests

Three new regression cases in model-selection.test.ts:

  • #when user primary model is unreachable and user fallback_models are provided#then promotes the first reachable user fallback
  • #then keeps the user primary when it IS reachable (fast path preserved)
  • #then returns the user primary as-is when no user fallback is reachable either (trust-user legacy behavior)

Verification

bun test src/tools/delegate-task/model-selection.test.ts
→ 25 pass, 0 fail

bun test src/tools/delegate-task/
→ 393 pass, 0 fail

bun run typecheck
→ clean

User impact

Users who observed hyperplan (or any team-mode category spawn) silently running with fewer members than requested — because one member's primary model wasn't reachable — will now see that member resolve via their configured fallbacks. No config change required on the user side; existing fallback_models entries now work as documented.


View in Codesmith
Need help on this PR? Tag @codesmith with what you need.

  • Let Codesmith autofix CI failures and bot reviews

Summary by cubic

Fixes model selection so user fallback_models are used when the category’s primary model isn’t available on connected providers. This prevents team members from spawning with unreachable models and restores graceful fallback.

  • Bug Fixes
    • In resolveModelForDelegateTask, when availability data exists and userFallbackModels are set, verify the primary; if unreachable, promote the first reachable fallback and set matchedFallback: true.
    • Preserve cold-cache behavior: return the user primary as-is when availability is unknown.
    • Add tests for: unreachable primary → reachable fallback, reachable primary retained, and both unreachable → primary kept.

Written for commit 711b753. Summary will update on new commits.

…is unreachable

resolveModelForDelegateTask returned input.userModel immediately without
availability checking when it was set, silently ignoring the user's
configured fallback_models. Effect: a team-mode category member
(e.g. hyperplan's 'artistry' category) configured with

  { model: "opencode/gemini-3.1-pro", fallback_models: [...] }

spawned with the unreachable primary even though a listed fallback was
reachable, leading to a dropped/broken team member instead of graceful
degradation.

Now, when the provider-models cache is warm AND userFallbackModels is
non-empty, the function verifies userModel is reachable via
fuzzyMatchModel. If not, it iterates userFallbackModels and promotes the
first reachable entry. Cold-cache (first-run) behavior is preserved -
the userModel is returned as-is when availability data is unavailable,
matching the existing 'trust the user' contract covered by the
pre-cache test fixtures.

Adds three regression tests covering: (1) unreachable primary + reachable
fallback -> fallback promoted, (2) reachable primary + reachable fallback
-> primary kept (fast path), (3) unreachable primary + unreachable
fallback -> legacy trust-user behavior preserved.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Auto-approved: The change is tightly scoped with three regression tests that verify all paths—primary reachable, fallback promoted when primary is unreachable, and fallback untouched when no fallback is reachable—while preserving cold-cache behavior, so there is no risk of regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants