fix(gateway): add video_generate to auto-append media allowlist#43065
Open
srojk34 wants to merge 1 commit into
Open
fix(gateway): add video_generate to auto-append media allowlist#43065srojk34 wants to merge 1 commit into
srojk34 wants to merge 1 commit into
Conversation
video_generate returns its artifact as JSON ({"success": true, "video":
"/abs/path.mp4"}) with no MEDIA: tag, so the gateway auto-append path
(which was gated on _AUTO_APPEND_MEDIA_TOOL_NAMES) never ran for it —
video delivery depended on the model restating the path in its reply.
Add video_generate to the allowlist and extract its local-file path from
the "video" field, mirroring the existing image_generate handler that
reads from ("host_image", "image", "agent_visible_image"). The same
_TOOL_MEDIA_RE guard applies: remote CDN URLs (which lack a local-path
prefix) are not matched and still require model restatement; local-path
backends benefit immediately.
Symmetric with the image_generate fix from NousResearch#42616.
Fixes NousResearch#19105 (video path)
13 tasks
Contributor
|
@srojk34 nice — this is the earlier and correct fix; I closed my dup #45786 in its favor. Two optional add-ons from that branch if useful: collapsing the image/video branches into a single per-tool field map (drops the duplicated handler block), and 5 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
video_generatewas missing from_AUTO_APPEND_MEDIA_TOOL_NAMESingateway/run.py. The gateway's auto-append logic is gated on that set,so it never ran for video tool results — video files were only delivered
if the model happened to restate the path in its reply text.
image_generatereceived the symmetric fix in #42616, butvideo_generatewas not included in the same change. This PR closes the remaining gap.
Root cause
video_generatereturns{"success": true, "video": "/abs/path.mp4", ...}with no
MEDIA:tag in its result string. Without an entry in_AUTO_APPEND_MEDIA_TOOL_NAMES,_collect_auto_append_media_tags()neverinspects the result, so no
MEDIA:tag is appended to the gateway reply.Fix
"video_generate"to_AUTO_APPEND_MEDIA_TOOL_NAMES._JSON_VIDEO_TOOL_PATH_FIELD = "video"constant (mirrors_JSON_MEDIA_TOOL_PATH_FIELDSfor the image path)._collect_auto_append_media_tags()thatJSON-parses the tool result, reads the
"video"field, and appends aMEDIA:tag when_TOOL_MEDIA_RE.fullmatch()passes (local-path guard).Remote CDN URLs (HTTPS) are not matched by
_TOOL_MEDIA_REand stillrequire model restatement — identical scope to the
image_generatefix.Fixes #19105 (video path).
Test plan
tests/gateway/test_media_extraction.py— 13/13 passvideo_generatewith a local-file backend:MEDIA:/abs/path.mp4appended automatically, video delivered without model restatement
video_generatewith a CDN-URL backend (FAL, xAI): noMEDIA:taginjected (URL not matched by
_TOOL_MEDIA_RE), behaviour unchangedimage_generatepath unaffected — no regression