Skip to content

Commit 72315cd

Browse files
SantiagoDePoloniaclaude
SantiagoDePolonia
and
claude
authored
Fix: multimodal chat content handling (#119)
* Fix multimodal chat content handling * Harden multimodal validation and guardrail handling * Fix multimodal schema and null-content handling * Update contract goldens for tool-call null content * Tighten multimodal schemas and request handling * Guard multimodal rewrite allocations * Fix responses schemas and multimodal guardrail merges * Preserve tool calls and adapter request options * Adjust README and CLAUDE guidance * Preserve responses tool controls in JSON * Fix responses JSON roundtrip and anthropic bounds * Harden multimodal guardrail and responses merges * Unify ContentPart types and add input_audio test coverage Eliminate duplicate ResponsesContentPart struct (identical to ContentPart) and use ContentPart everywhere. Add 11 tests covering input_audio unmarshaling, normalization, validation, and mixed content scenarios. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix multimodal guardrail and responses adaptation * Clean up duplicate code, dead code, and add test coverage - Deduplicate isNullEquivalentContent (was two identical functions) - Remove unreachable trailing loop in applySystemMessagesToMultimodalChat - Remove unused anthropicMessageContentBlock type alias - Simplify single-variable var block - Add test for input_text → text normalization in Anthropic conversion - Add test for maxContentParts overflow guard Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Enable dupl linter and deduplicate GetConversation logic Add dupl linter (threshold 150, excluded for test files) to catch duplicate code blocks. Extract shared conversation thread-walking logic into buildConversationThread helper used by both MongoDB and PostgreSQL readers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Replace ResponsesInputItem with discriminated union ResponsesInputElement The old ResponsesInputItem only modeled chat messages (role + content), forcing function_call and function_call_output items through untyped map[string]interface{} handling. The new ResponsesInputElement struct explicitly represents all three input variants via a Type discriminator, with typed fields for each shape. JSON unmarshal now produces []ResponsesInputElement instead of []interface{}, eliminating map[string]interface{} from the hot path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 7ff630a commit 72315cd

27 files changed

Lines changed: 4064 additions & 861 deletions

‎.golangci.yml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,18 @@
11
version: "2"
22
linters:
3+
enable:
4+
- dupl
5+
settings:
6+
dupl:
7+
threshold: 150
38
exclusions:
9+
rules:
10+
- linters:
11+
- dupl
12+
path: _test\.go
13+
- linters:
14+
- dupl
15+
path: tests/
416
generated: lax
517
presets:
618
- comments

‎CLAUDE.md‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -164,9 +164,8 @@ Full reference: `.env.template` and `config/config.yaml`
164164

165165
After completing any code change, routinely check whether documentation needs updating. This applies to all three documentation layers:
166166

167-
1. **README files** (`README.md`, `helm/README.md`, `tests/contract/README.md`) — Update when adding/removing features, changing setup steps, modifying CLI flags, or altering configuration options.
168-
2. **In-code documentation** (Go doc comments on exported types, functions, interfaces) — Update when changing public APIs, adding new exported symbols, or modifying function signatures/behavior.
169-
3. **Mintlify / technical docs** (`docs/` directory) — Update `docs/advanced/*.mdx` pages when changing configuration options or guardrails behavior. Update `docs/adr/` when making significant architectural decisions. Update `docs/plans/` if implementation diverges from existing plans. Check `docs.json` if new pages need to be added to the navigation.
167+
1. **In-code documentation** (Go doc comments on exported types, functions, interfaces) — Update when changing public APIs, adding new exported symbols, or modifying function signatures/behavior.
168+
2. **Mintlify / technical docs** (`docs/` directory) — Update `docs/advanced/*.mdx` pages when changing configuration options or guardrails behavior. Update `docs/adr/` when making significant architectural decisions. Update `docs/plans/` if implementation diverges from existing plans. Check `docs.json` if new pages need to be added to the navigation.
170169

171170
**When to update:**
172171
- Adding a new provider, endpoint, config option, or feature

‎cmd/gomodel/docs/docs.go‎

Lines changed: 97 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1848,7 +1848,7 @@ const docTemplate = `{
18481848
"type": "integer"
18491849
},
18501850
"message": {
1851-
"$ref": "#/definitions/core.Message"
1851+
"$ref": "#/definitions/core.ResponseMessage"
18521852
}
18531853
}
18541854
},
@@ -1869,6 +1869,23 @@ const docTemplate = `{
18691869
}
18701870
}
18711871
},
1872+
"core.ContentPart": {
1873+
"type": "object",
1874+
"properties": {
1875+
"image_url": {
1876+
"$ref": "#/definitions/core.ImageURLContent"
1877+
},
1878+
"input_audio": {
1879+
"$ref": "#/definitions/core.InputAudioContent"
1880+
},
1881+
"text": {
1882+
"type": "string"
1883+
},
1884+
"type": {
1885+
"type": "string"
1886+
}
1887+
}
1888+
},
18721889
"core.EmbeddingData": {
18731890
"type": "object",
18741891
"properties": {
@@ -2044,11 +2061,41 @@ const docTemplate = `{
20442061
}
20452062
}
20462063
},
2064+
"core.ImageURLContent": {
2065+
"type": "object",
2066+
"properties": {
2067+
"detail": {
2068+
"type": "string"
2069+
},
2070+
"media_type": {
2071+
"type": "string"
2072+
},
2073+
"url": {
2074+
"type": "string"
2075+
}
2076+
}
2077+
},
2078+
"core.InputAudioContent": {
2079+
"type": "object",
2080+
"properties": {
2081+
"data": {
2082+
"type": "string"
2083+
},
2084+
"format": {
2085+
"type": "string"
2086+
}
2087+
}
2088+
},
20472089
"core.Message": {
20482090
"type": "object",
20492091
"properties": {
20502092
"content": {
2051-
"type": "string"
2093+
"description": "ContentSchema documents that ` + "`" + `content` + "`" + ` accepts either a plain string\nor an array of ContentPart values.",
2094+
"type": "array",
2095+
"items": {
2096+
"$ref": "#/definitions/core.ContentPart"
2097+
},
2098+
"x-oneof": "[{\"type\":\"null\"},{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ContentPart\"}}]"
20522099
},
20532100
"role": {
20542101
"type": "string"
@@ -2273,6 +2320,27 @@ const docTemplate = `{
22732320
}
22742321
}
22752322
},
2323+
"core.ResponseMessage": {
2324+
"type": "object",
2325+
"properties": {
2326+
"content": {
2327+
"type": "array",
2328+
"items": {
2329+
"$ref": "#/definitions/core.ContentPart"
2330+
},
2331+
"x-oneof": "[{\"type\":\"null\"},{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ContentPart\"}}]"
2332+
},
2333+
"role": {
2334+
"type": "string"
2335+
},
2336+
"tool_calls": {
2337+
"type": "array",
2338+
"items": {
2339+
"$ref": "#/definitions/core.ToolCall"
2340+
}
2341+
}
2342+
}
2343+
},
22762344
"core.ResponsesContentItem": {
22772345
"type": "object",
22782346
"properties": {
@@ -2282,11 +2350,17 @@ const docTemplate = `{
22822350
"type": "string"
22832351
}
22842352
},
2353+
"image_url": {
2354+
"$ref": "#/definitions/core.ImageURLContent"
2355+
},
2356+
"input_audio": {
2357+
"$ref": "#/definitions/core.InputAudioContent"
2358+
},
22852359
"text": {
22862360
"type": "string"
22872361
},
22882362
"type": {
2289-
"description": "\"output_text\", etc.",
2363+
"description": "\"output_text\", \"input_image\", \"input_audio\", etc.",
22902364
"type": "string"
22912365
}
22922366
}
@@ -2302,6 +2376,21 @@ const docTemplate = `{
23022376
}
23032377
}
23042378
},
2379+
"core.ResponsesInputItem": {
2380+
"type": "object",
2381+
"properties": {
2382+
"content": {
2383+
"type": "array",
2384+
"items": {
2385+
"$ref": "#/definitions/core.ContentPart"
2386+
},
2387+
"x-oneof": "[{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ContentPart\"}}]"
2388+
},
2389+
"role": {
2390+
"type": "string"
2391+
}
2392+
}
2393+
},
23052394
"core.ResponsesOutputItem": {
23062395
"type": "object",
23072396
"properties": {
@@ -2339,9 +2428,11 @@ const docTemplate = `{
23392428
"type": "object",
23402429
"properties": {
23412430
"input": {
2342-
"description": "string or []ResponsesInputItem — see docs for array form",
2343-
"type": "string",
2344-
"example": "Tell me a joke"
2431+
"type": "array",
2432+
"items": {
2433+
"$ref": "#/definitions/core.ResponsesInputItem"
2434+
},
2435+
"x-oneof": "[{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ResponsesInputItem\"}}]"
23452436
},
23462437
"instructions": {
23472438
"type": "string"

‎cmd/gomodel/docs/swagger.json‎

Lines changed: 97 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1844,7 +1844,7 @@
18441844
"type": "integer"
18451845
},
18461846
"message": {
1847-
"$ref": "#/definitions/core.Message"
1847+
"$ref": "#/definitions/core.ResponseMessage"
18481848
}
18491849
}
18501850
},
@@ -1865,6 +1865,23 @@
18651865
}
18661866
}
18671867
},
1868+
"core.ContentPart": {
1869+
"type": "object",
1870+
"properties": {
1871+
"image_url": {
1872+
"$ref": "#/definitions/core.ImageURLContent"
1873+
},
1874+
"input_audio": {
1875+
"$ref": "#/definitions/core.InputAudioContent"
1876+
},
1877+
"text": {
1878+
"type": "string"
1879+
},
1880+
"type": {
1881+
"type": "string"
1882+
}
1883+
}
1884+
},
18681885
"core.EmbeddingData": {
18691886
"type": "object",
18701887
"properties": {
@@ -2040,11 +2057,41 @@
20402057
}
20412058
}
20422059
},
2060+
"core.ImageURLContent": {
2061+
"type": "object",
2062+
"properties": {
2063+
"detail": {
2064+
"type": "string"
2065+
},
2066+
"media_type": {
2067+
"type": "string"
2068+
},
2069+
"url": {
2070+
"type": "string"
2071+
}
2072+
}
2073+
},
2074+
"core.InputAudioContent": {
2075+
"type": "object",
2076+
"properties": {
2077+
"data": {
2078+
"type": "string"
2079+
},
2080+
"format": {
2081+
"type": "string"
2082+
}
2083+
}
2084+
},
20432085
"core.Message": {
20442086
"type": "object",
20452087
"properties": {
20462088
"content": {
2047-
"type": "string"
2089+
"description": "ContentSchema documents that `content` accepts either a plain string\nor an array of ContentPart values.",
2090+
"type": "array",
2091+
"items": {
2092+
"$ref": "#/definitions/core.ContentPart"
2093+
},
2094+
"x-oneof": "[{\"type\":\"null\"},{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ContentPart\"}}]"
20482095
},
20492096
"role": {
20502097
"type": "string"
@@ -2269,6 +2316,27 @@
22692316
}
22702317
}
22712318
},
2319+
"core.ResponseMessage": {
2320+
"type": "object",
2321+
"properties": {
2322+
"content": {
2323+
"type": "array",
2324+
"items": {
2325+
"$ref": "#/definitions/core.ContentPart"
2326+
},
2327+
"x-oneof": "[{\"type\":\"null\"},{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ContentPart\"}}]"
2328+
},
2329+
"role": {
2330+
"type": "string"
2331+
},
2332+
"tool_calls": {
2333+
"type": "array",
2334+
"items": {
2335+
"$ref": "#/definitions/core.ToolCall"
2336+
}
2337+
}
2338+
}
2339+
},
22722340
"core.ResponsesContentItem": {
22732341
"type": "object",
22742342
"properties": {
@@ -2278,11 +2346,17 @@
22782346
"type": "string"
22792347
}
22802348
},
2349+
"image_url": {
2350+
"$ref": "#/definitions/core.ImageURLContent"
2351+
},
2352+
"input_audio": {
2353+
"$ref": "#/definitions/core.InputAudioContent"
2354+
},
22812355
"text": {
22822356
"type": "string"
22832357
},
22842358
"type": {
2285-
"description": "\"output_text\", etc.",
2359+
"description": "\"output_text\", \"input_image\", \"input_audio\", etc.",
22862360
"type": "string"
22872361
}
22882362
}
@@ -2298,6 +2372,21 @@
22982372
}
22992373
}
23002374
},
2375+
"core.ResponsesInputItem": {
2376+
"type": "object",
2377+
"properties": {
2378+
"content": {
2379+
"type": "array",
2380+
"items": {
2381+
"$ref": "#/definitions/core.ContentPart"
2382+
},
2383+
"x-oneof": "[{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ContentPart\"}}]"
2384+
},
2385+
"role": {
2386+
"type": "string"
2387+
}
2388+
}
2389+
},
23012390
"core.ResponsesOutputItem": {
23022391
"type": "object",
23032392
"properties": {
@@ -2335,9 +2424,11 @@
23352424
"type": "object",
23362425
"properties": {
23372426
"input": {
2338-
"description": "string or []ResponsesInputItem — see docs for array form",
2339-
"type": "string",
2340-
"example": "Tell me a joke"
2427+
"type": "array",
2428+
"items": {
2429+
"$ref": "#/definitions/core.ResponsesInputItem"
2430+
},
2431+
"x-oneof": "[{\"type\":\"string\"},{\"type\":\"array\",\"items\":{\"$ref\":\"#/definitions/core.ResponsesInputItem\"}}]"
23412432
},
23422433
"instructions": {
23432434
"type": "string"

0 commit comments

Comments
 (0)