fix(tokens): catch pre-underscore prefix in silent reply detection by ninjaa · Pull Request #34733 · openclaw/openclaw

ninjaa · 2026-03-04T16:58:44Z

Problem

During streaming, when a model outputs NO_REPLY (the silent reply token), the first chunk "NO" can arrive before "_REPLY". The prefix checker isSilentReplyPrefixText() requires an underscore to be present, so "NO" alone fails the check and gets delivered to the channel as a visible message.

Evidence: 9 leaked "NO" messages across multiple Mattermost channels. Full NO_REPLY never leaks — only "NO".

Root Cause

// tokens.ts — old logic
if (!normalized.includes("_")) return false;  // ← "NO" fails here

The underscore requirement was added to avoid false positives ("N", "No" are common words), but it creates a window where the pre-underscore fragment leaks during streaming.

Fix

Generalize the check: accept the exact pre-underscore segment of any silent token as a valid prefix. For NO_REPLY, "NO" now matches. For HEARTBEAT_OK, "HEARTBEAT" matches. Partial segments like "N", "HE", "HEART" still correctly return false.

4 lines changed in the function. Works for any WORD_SUFFIX style token.

Tests

Added T621-specific test block covering all edge cases. Updated existing generic token test. All 20 token tests pass.

Fixes #32168
Closes #32403

…621) During streaming, 'NO' can arrive as a transient fragment before '_REPLY'. The existing isSilentReplyPrefixText() required an underscore to be present, so 'NO' slipped through and leaked to channels. Generalize the check: accept the exact pre-underscore segment of any silent token as a valid prefix (e.g. 'NO' for NO_REPLY, 'HEARTBEAT' for HEARTBEAT_OK). Fixes openclaw#32168 Closes openclaw#32403

greptile-apps · 2026-03-04T17:00:50Z

Greptile Summary

This PR fixes a streaming leak bug where the pre-underscore fragment of a silent reply token (e.g. "NO" from "NO_REPLY") would bypass the isSilentReplyPrefixText check and be delivered as a visible message. The fix replaces a hardcoded "NO" special-case with a generalized rule: accept the exact pre-underscore segment of any WORD_SUFFIX-style token as a valid prefix.

Key changes:

tokens.ts: Replaces return tokenUpper === SILENT_REPLY_TOKEN && normalized === "NO" with a general tokenUpper.indexOf("_")-based check that works for any token (e.g. "HEARTBEAT" for "HEARTBEAT_OK"). All existing guards (all-uppercase, length ≥ 2, only A–Z and underscores, must be a prefix of the token) are preserved.
tokens.test.ts: Updates the old assertion where "HEARTBEAT" was expected to return false with "HEARTBEAT_OK" (now correctly true), and adds a T621-specific block. Note: the T621 block largely duplicates assertions already present in earlier describe blocks — consider trimming it to only the novel case ("NX" → false).

Confidence Score: 5/5

This PR is safe to merge — the fix is minimal, logically sound, and well-tested.
The change is 4 lines in one function, replacing a hardcoded special-case with a correct generalization. All existing guards remain intact, so no false-positive risk is introduced. The tests pass and cover the key streaming-fragment scenarios. The only finding is minor test duplication, which does not affect correctness.
No files require special attention.

_{Last reviewed commit: 26bd813}

src/auto-reply/tokens.test.ts

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26bd813ea6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-04T17:01:49Z

src/auto-reply/tokens.ts

+  const underscoreIdx = tokenUpper.indexOf("_");
+  if (underscoreIdx > 0 && normalized === tokenUpper.slice(0, underscoreIdx)) {
+    return true;


Preserve underscore guard for non-silent sentinel tokens

This new branch makes "HEARTBEAT" match HEARTBEAT_OK as a silent-prefix, which creates a false positive in production paths that call isSilentReplyPrefixText(text, HEARTBEAT_TOKEN) (see src/auto-reply/reply/agent-runner-execution.ts:154-155): if a model legitimately emits the uppercase word HEARTBEAT (for example when asked to output that token verbatim), the chunk is now suppressed as control text and can be dropped from user-visible output. The previous underscore guard avoided suppressing plain uppercase words for non-NO_REPLY tokens.

Useful? React with 👍 / 👎.

This is intentional. For isSilentReplyPrefixText to suppress "HEARTBEAT", ALL of these must hold simultaneously:

The model's entire accumulated output is exactly HEARTBEAT (all caps, no spaces, no other chars)

More tokens are still streaming (if final, isSilentReplyText handles it separately)

The model was producing this bare word as its complete response start

This is not a realistic production scenario. Models don't output bare HEARTBEAT as replies. Even if they did, suppression is transient — it only holds for one streaming tick until the next token arrives and breaks the prefix match (e.g., "HEARTBEAT " has a space → fails [^A-Z_] → flushes immediately).

The previous underscore guard was the root cause of the bug: it was too narrow, only catching "NO" for NO_REPLY while leaving the same leak vector open for any future tokens. The generalized approach trades an astronomically unlikely false positive for complete leak prevention across all silent token variants.

…egment Per review: buffer N, NO, and any uppercase prefix up to the underscore position, not just the exact pre-underscore word. This closes the edge case where a tokenizer might emit a single char like 'N' before 'O_REPLY'. Safe because: text must be strictly uppercase, an exact prefix of the token, and contain only A-Z_ chars. Mixed-case ('No', 'no') is rejected.

ninjaa · 2026-03-04T17:10:09Z

Updated: Now buffers all uppercase prefixes of the token, not just the exact pre-underscore segment.

How the holdback works

isSilentReplyPrefixText is checked at the streaming delivery boundary — when accumulated text would be sent to the channel. If it returns true, the text is held back (skip: true). On the next streaming tick:

If more tokens complete NO_REPLY → isSilentReplyText catches it, message silently dropped ✅
If tokens diverge (e.g. "NOTICE...") → no longer a prefix, text flushes to channel ✅

What gets buffered now

Fragment	Buffered?	Why
`"N"`	✅ yes	Strict uppercase prefix of `NO_REPLY`
`"NO"`	✅ yes	The actual observed leak (9 instances)
`"NO_"`	✅ yes	Contains underscore, existing logic
`"NO_RE"`	✅ yes	Contains underscore, existing logic
`"NO_REPLY"`	✅ yes	Exact match → silently dropped
`"No"`	❌ no	Mixed case → real text
`"NOPE"`	❌ no	Not a prefix of `NO_REPLY`
`"N/A"`	❌ no	Contains `/`, fails `[^A-Z_]` check

Why this is safe

The function requires ALL of:

Text is trimmed and strictly uppercase (mixed case like "No" → instant reject)
Text contains only A-Z and _ characters
Text is an exact prefix of the token

A real model reply starting with a single uppercase "N" would almost immediately get a lowercase/space/punctuation character on the next token, at which point it stops matching and flushes.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6eba6ad55

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-04T17:13:41Z

src/auto-reply/tokens.ts

+  if (underscoreIdx > 0 && normalized.length <= underscoreIdx) {
+    return true;


Keep pre-underscore prefix matching scoped to NO_REPLY

This branch now marks any pre-underscore prefix as silent (HEARTBEAT, HE, etc.), which regresses non-silent token handling because normalizeStreamingText calls isSilentReplyPrefixText(text, HEARTBEAT_TOKEN) in agent-runner-execution.ts and will skip those payloads as control text. Fresh evidence: that same normalizer is also used by block reply delivery, so in block-streaming runs these suppressed chunks can be dropped from user-visible output rather than just hidden from typing signals. Please keep this relaxed pre-underscore rule limited to NO_REPLY (or an explicit allowlist) instead of applying it to all underscore tokens.

Useful? React with 👍 / 👎.

openclaw-barnacle bot added the size: XS label Mar 4, 2026

greptile-apps bot reviewed Mar 4, 2026

View reviewed changes

src/auto-reply/tokens.test.ts Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 4, 2026

View reviewed changes

openclaw-barnacle bot added size: S and removed size: XS labels Mar 4, 2026

chatgpt-codex-connector bot reviewed Mar 4, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(tokens): catch pre-underscore prefix in silent reply detection#34733

fix(tokens): catch pre-underscore prefix in silent reply detection#34733
ninjaa wants to merge 2 commits intoopenclaw:mainfrom
ninjaa:fix/silent-reply-holdback-buffer-t621

ninjaa commented Mar 4, 2026

Uh oh!

greptile-apps bot commented Mar 4, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Mar 4, 2026

Uh oh!

ninjaa Mar 4, 2026

Uh oh!

ninjaa commented Mar 4, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Mar 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		if (underscoreIdx > 0 && normalized.length <= underscoreIdx) {
		return true;

Uh oh!

Conversation

ninjaa commented Mar 4, 2026

Problem

Root Cause

Fix

Tests

Uh oh!

greptile-apps bot commented Mar 4, 2026

Greptile Summary

Confidence Score: 5/5

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

ninjaa Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

ninjaa commented Mar 4, 2026

How the holdback works

What gets buffered now

Why this is safe

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant