fix(vision): Z.AI (智谱 GLM) vision model compatibility#19346
Closed
agilejava wants to merge 1 commit into
Closed
Conversation
…ax_tokens handling
Z.AI (智谱 GLM) vision models (glm-4v-flash, glm-4v-plus, etc.) have two
compatibility issues when used through the Anthropic-compatible endpoint:
1. **Error 1210 — max_tokens rejected on multimodal calls**: Z.AI rejects
the max_tokens parameter for vision model requests with error code 1210
("API 调用参数有误"). The error string does not contain "max_tokens",
so the existing unsupported-parameter retry logic never fires.
2. **Wrong endpoint inheritance**: When the main runtime provider uses Z.AI's
Anthropic-compatible endpoint (open.bigmodel.cn/api/anthropic), the vision
client inherits this endpoint. But Z.AI's Anthropic wire cannot properly
handle image content — models silently fail ("I can't see the image") or
reject max_tokens.
Changes:
- resolve_vision_provider_client(): force Z.AI vision to use OpenAI-compatible
endpoint (open.bigmodel.cn/api/paas/v4) instead of inheriting Anthropic wire
- _build_call_kwargs(): skip max_tokens for Z.AI vision models (4v/5v/-v suffix)
- _AnthropicCompletionsAdapter: support _skip_zai_max_tokens flag
- _to_openai_base_url(): rewrite Z.AI Anthropic URLs to OpenAI-compatible path
- call_llm() retry: detect Z.AI error 1210 and strip max_tokens before retry
This was referenced May 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Z.AI (智谱 GLM) vision models (
glm-4v-flash,glm-4v-plus,glm-4v, etc.) fail when used as theauxiliary.visionprovider due to two compatibility issues:Issue 1: Error 1210 — max_tokens rejected on multimodal calls
Z.AI rejects the
max_tokensparameter for vision model requests with error code 1210 ("API 调用参数有误"). The error string does not contain"max_tokens", so the existing unsupported-parameter retry logic incall_llm()never fires. This results in a hard failure with no recovery.Issue 2: Wrong endpoint inheritance (silent failure)
When the main runtime provider uses Z.AI's Anthropic-compatible endpoint (
open.bigmodel.cn/api/anthropic), the vision client inherits this endpoint viaresolve_vision_provider_client(). Z.AI's Anthropic wire cannot properly handle image content — models silently fail with responses like"I can't see the image"(received base64 data but couldn't process it through the Anthropic translation layer). This is a silent failure with no HTTP error, making it extremely hard to diagnose.Changes
resolve_vision_provider_client()— Force Z.AI vision to use the OpenAI-compatible endpoint (open.bigmodel.cn/api/paas/v4) instead of inheriting the Anthropic wire from the main runtime provider._build_call_kwargs()— Skipmax_tokensfor Z.AI vision models (detected by4v/5v/-vsuffix in model name)._AnthropicCompletionsAdapter— Support_skip_zai_max_tokensflag for cases where the adapter is still used._to_openai_base_url()— Rewrite Z.AI Anthropic URLs to OpenAI-compatible path (handles edge case URL rewriting).call_llm()retry — Detect Z.AI-specific error 1210 and stripmax_tokensbefore retrying.Testing
glm-4v-flashon Z.AI free tier — vision analysis works correctlyvision_analyzetool returns correct descriptions of test imagesConfig used for testing
Main provider:
zaiwithbase_url: https://open.bigmodel.cn/api/anthropic