fix(honcho): flatten multimodal list content in sync_turn (#30252)#30294
Open
Bartok9 wants to merge 1 commit into
Open
fix(honcho): flatten multimodal list content in sync_turn (#30252)#30294Bartok9 wants to merge 1 commit into
Bartok9 wants to merge 1 commit into
Conversation
Collaborator
|
Competing fix with #30264, #22768, and #26484 — all add multimodal list content flattening to Honcho |
Contributor
Author
|
Closing to stay within contributor PR limit. Will resubmit with fresh rebase if the issue remains open in main. |
…ch#30252) OpenAI-style multimodal user messages carry content as a list: [{"type": "text", "text": "..."}, {"type": "image_url", ...}] Previously, sync_turn called sanitize_context(user_content or "") directly on this list value, causing: TypeError: expected string or bytes-like object, got 'list' The exception was silently swallowed by the _sync() wrapper and the vision turn was dropped from Honcho's memory entirely. Fix: add a _flatten_content() static method that: - passes plain strings through unchanged - joins text parts with a space - replaces image_url/image blocks with the literal placeholder [image] - uses a generic [<type>] placeholder for any unknown block types sync_turn now accepts Union[str, list, None] for both arguments and routes them through _flatten_content() before sanitise_context(). Fixes NousResearch#30252
d34f7d3 to
b281340
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
HonchoMemoryProvider.sync_turnerrors when called with OpenAI-style multimodal list content (the shape used for vision turns):The call
sanitize_context(user_content or "")receives alist, raising:The exception is caught silently by the
_sync()wrapper and the turn is silently dropped from Honcho's memory — the user representation never learns about anything visual the assistant was shown.Closes #30252.
Fix
Add a
_flatten_content()static method that normalises list-shaped content to a plain string before sanitisation:textparts are joined with a spaceimage_url/imageparts become the literal placeholder[image][<type>]sync_turnnow acceptsstr | list | Nonefor both arguments.Tests
5 new tests in
TestSyncTurnMultimodalContent:[image]placeholderNonecontent does not raise_flatten_contentstatic method handles all known block typesAll 120 existing honcho plugin tests pass.