fix(cli): preserve model-specific api_mode across credential refresh#17596
Open
toller892 wants to merge 1 commit into
Open
fix(cli): preserve model-specific api_mode across credential refresh#17596toller892 wants to merge 1 commit into
toller892 wants to merge 1 commit into
Conversation
_ensure_runtime_credentials() unconditionally overwrites self.api_mode with resolve_runtime_provider()'s return value, which defaults to "chat_completions" for most providers. This destroys the transport mode set by the /model command for models that require non-default transports (e.g. kimi-for-coding needs "anthropic_messages"). Fix: only overwrite api_mode when the resolver returns a non-default value, or when we're already at the default. This preserves model-specific transport modes across credential refreshes. Fixes NousResearch#17574
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_ensure_runtime_credentials()unconditionally overwritesself.api_modewithresolve_runtime_provider()s return value on every turn. The resolver defaults to"chat_completions"for most providers, but some models require"anthropic_messages"(e.g. kimi-for-coding).When a user runs
/model kimi-for-coding, it correctly setsapi_mode="anthropic_messages". But on the next turn,_ensure_runtime_credentials()resets it back to"chat_completions", causing a 404 error (OpenAI-format traffic sent to an Anthropic-format endpoint).Repro
/model kimi-for-codingRoot Cause
resolve_runtime_provider()always returns anapi_mode(defaulting to"chat_completions"), so the fallbackruntime.get("api_mode", self.api_mode)never usesself.api_mode. The/modelcommands correctly set transport is unconditionally overwritten.Fix
Only overwrite
self.api_modewhen:"chat_completions") — nothing to preserveThis preserves model-specific transport modes set by
/modelwhile still allowing the resolver to communicate non-default transports.Related
Fixes #17574