fix(agent): prevent GLM stop-to-length heuristic false positives (#14572)#25991
Closed
aj-nt wants to merge 1 commit into
Closed
fix(agent): prevent GLM stop-to-length heuristic false positives (#14572)#25991aj-nt wants to merge 1 commit into
aj-nt wants to merge 1 commit into
Conversation
…sResearch#14572) The Ollama/GLM stop-to-length heuristic in _should_treat_stop_as_truncated() produced false positives for agents using emoji sign-offs and short conversational replies. Three changes: 1. _has_natural_response_ending() now recognises emoji and Unicode symbols as natural endings using unicodedata categories (So/Sk/Sm) and the Extended_Pictographic codepoint range. Also strips trailing Unicode combining marks (variation selectors, ZWJ) before checking the last real glyph so emoji+VS16 is recognised correctly. 2. _should_treat_stop_as_truncated() enforces a 500-character minimum gate on visible text. Short replies are almost certainly not truncated and cannot hit meaningful token limits. 3. New config flag agent.glm_truncation_heuristic (default true) lets users opt out of the heuristic entirely. Includes 17 new tests covering emoji/symbol recognition, 500-char gate behaviour, and integration-level short-response detection.
Collaborator
Author
|
Closing in favor of #14574 which was submitted first. Will contribute the 500-char gate and config opt-out as suggestions there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Ollama/GLM stop-to-length heuristic in
_should_treat_stop_as_truncated()produces false positives for agents using emoji sign-offs (💛 ✨ 🙌) and short conversational replies, wasting up to 60 turns per user message.Fix
Three changes:
_has_natural_response_ending()now recognises emoji and Unicode symbols as natural endings usingunicodedatacategories (So/Sk/Sm) and the Extended_Pictographic codepoint range. Also strips trailing Unicode combining marks (variation selectors, ZWJ) before checking the last real glyph so emoji+VS16 is recognised correctly._should_treat_stop_as_truncated()enforces a 500-character minimum gate on visible text. Short replies are almost certainly not truncated and cannot hit meaningful token limits.Config opt-out: new
agent.glm_truncation_heuristicflag (defaulttrue) lets users disable the heuristic entirely.Changes
run_agent.py: Expanded_has_natural_response_ending()(+42 lines) and_should_treat_stop_as_truncated()(+18 lines) with config opt-outcli-config.yaml.example: Documented new config flagtests/run_agent/test_glm_stop_heuristic.py: 17 new tests (emoji/symbol recognition, 500-char gate, integration)tests/run_agent/test_run_agent.py: Updated existing test for 500-char gateTesting
Closes #14572