Skip to content

fix: strip <thought> tags from Gemma 4 responses in _strip_think_blocks#6149

Closed
Unayung wants to merge 2 commits into
NousResearch:mainfrom
Unayung:fix/strip-gemma4-thought-tags
Closed

fix: strip <thought> tags from Gemma 4 responses in _strip_think_blocks#6149
Unayung wants to merge 2 commits into
NousResearch:mainfrom
Unayung:fix/strip-gemma4-thought-tags

Conversation

@Unayung

@Unayung Unayung commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Gemma 4 (26B/31B) wraps its reasoning output in <thought>...</thought> tags. This tag variant was not handled by _strip_think_blocks(), causing raw thinking blocks to leak into the visible response when using these models via the Gemini API.

Changes

Added <thought> to the list of stripped reasoning tag variants in _strip_think_blocks() (run_agent.py):

# Before
content = re.sub(r'</?(?:think|thinking|reasoning|REASONING_SCRATCHPAD)>\s*', '', ...)

# After
content = re.sub(r'<thought>.*?</thought>', '', content, flags=re.DOTALL | re.IGNORECASE)
content = re.sub(r'</?(?:think|thinking|reasoning|thought|REASONING_SCRATCHPAD)>\s*', '', ...)

Testing

Configure a Gemma 4 26B or 31B model via the Gemini API and send a message that triggers reasoning — <thought> blocks should no longer appear in the response.

Fixes #6148

@Acters

Acters commented Apr 9, 2026

Copy link
Copy Markdown

LGTM, I installed this, and it works with Gemma 4. I don't believe there is more testing required.
Thank you, these models are dam good. Pretty much at GPT-5.4-mini level and smarter than Gemini-3-flash.
hoping this gets merged soon

Gemma 4 (26B/31B) uses <thought>...</thought> to wrap its reasoning
output. This tag was not included in the existing list of reasoning tag
variants stripped by _strip_think_blocks(), causing raw thinking blocks
to leak into the visible response.

Added a new re.sub() line for <thought> and extended the cleanup regex
to include 'thought' alongside the existing variants.

Fixes NousResearch#6148
@Unayung Unayung force-pushed the fix/strip-gemma4-thought-tags branch from 09a0aec to 35c70ae Compare April 9, 2026 01:37
@lhear

lhear commented Apr 10, 2026

Copy link
Copy Markdown

Sometimes the model leaks thought tags or internal reasoning into the final output.

@vdsmon

vdsmon commented Apr 11, 2026

Copy link
Copy Markdown

Good fix! I hit this same issue with Gemma 4 31B via the Gemini API and applied the same patch locally.

One thing I noticed: _extract_reasoning() (around line 1721) also has a hardcoded list of inline patterns used to capture reasoning content when no structured reasoning fields are present:

inline_patterns = (
    r"<think>(.*?)</think>",
    r"<thinking>(.*?)</thinking>",
    r"<reasoning>(.*?)</reasoning>",
    r"<REASONING_SCRATCHPAD>(.*?)</REASONING_SCRATCHPAD>",
)

<thought> is missing here too. Without it, Gemma 4 reasoning gets silently discarded instead of being routed to the reasoning display (when show_reasoning: true). The fix in _strip_think_blocks prevents the leak, but the reasoning data is lost instead of properly captured.

Suggested addition:

inline_patterns = (
    r"<think>(.*?)</think>",
    r"<thinking>(.*?)</thinking>",
    r"<thought>(.*?)</thought>",      # Gemma 4
    r"<reasoning>(.*?)</reasoning>",
    r"<REASONING_SCRATCHPAD>(.*?)</REASONING_SCRATCHPAD>",
)

Probably has to do with what @lhear reported.

@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #8562. Your commits were cherry-picked onto current main with authorship preserved. Added the missing auxiliary_client.py fix and tests on top. Thanks @Unayung!

@teknium1 teknium1 closed this Apr 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: <thought> tags from Gemma 4 not stripped from response content

5 participants