fix: strip <thought> tags from Gemma 4 responses in _strip_think_blocks#6149
fix: strip <thought> tags from Gemma 4 responses in _strip_think_blocks#6149Unayung wants to merge 2 commits into
Conversation
|
LGTM, I installed this, and it works with Gemma 4. I don't believe there is more testing required. |
Gemma 4 (26B/31B) uses <thought>...</thought> to wrap its reasoning output. This tag was not included in the existing list of reasoning tag variants stripped by _strip_think_blocks(), causing raw thinking blocks to leak into the visible response. Added a new re.sub() line for <thought> and extended the cleanup regex to include 'thought' alongside the existing variants. Fixes NousResearch#6148
09a0aec to
35c70ae
Compare
|
Sometimes the model leaks thought tags or internal reasoning into the final output. |
|
Good fix! I hit this same issue with Gemma 4 31B via the Gemini API and applied the same patch locally. One thing I noticed: inline_patterns = (
r"<think>(.*?)</think>",
r"<thinking>(.*?)</thinking>",
r"<reasoning>(.*?)</reasoning>",
r"<REASONING_SCRATCHPAD>(.*?)</REASONING_SCRATCHPAD>",
)
Suggested addition: inline_patterns = (
r"<think>(.*?)</think>",
r"<thinking>(.*?)</thinking>",
r"<thought>(.*?)</thought>", # Gemma 4
r"<reasoning>(.*?)</reasoning>",
r"<REASONING_SCRATCHPAD>(.*?)</REASONING_SCRATCHPAD>",
)Probably has to do with what @lhear reported. |
Summary
Gemma 4 (26B/31B) wraps its reasoning output in
<thought>...</thought>tags. This tag variant was not handled by_strip_think_blocks(), causing raw thinking blocks to leak into the visible response when using these models via the Gemini API.Changes
Added
<thought>to the list of stripped reasoning tag variants in_strip_think_blocks()(run_agent.py):Testing
Configure a Gemma 4 26B or 31B model via the Gemini API and send a message that triggers reasoning —
<thought>blocks should no longer appear in the response.Fixes #6148