fix(session-search): strip FTS5 operators from truncation query terms#18692
Closed
liuhao1024 wants to merge 1 commit into
Closed
fix(session-search): strip FTS5 operators from truncation query terms#18692liuhao1024 wants to merge 1 commit into
liuhao1024 wants to merge 1 commit into
Conversation
When _truncate_around_matches receives an FTS5 query with boolean
operators (AND, OR, NOT), those operators were split into individual
terms and searched for in the conversation text. Since common English
words like 'and' appear everywhere, this produced noisy match positions
and mis-centered truncation windows.
Add _strip_fts5_operators() helper that removes:
- Boolean operators (AND, OR, NOT) — case-insensitive
- NEAR(...) clauses
- Column filters (e.g. role:user)
- FTS5 special characters (+, {}, (), ^, ~, *)
- Quoted-phrase delimiters (content preserved)
The cleaned query is used for all three matching strategies (phrase,
proximity co-occurrence, individual terms).
Fixes NousResearch#4238, NousResearch#4239
Collaborator
Cyrene963
pushed a commit
to Cyrene963/hermes-agent
that referenced
this pull request
May 3, 2026
Community PRs applied: - NousResearch#18596: Enable secret redaction by default (SECURITY) - NousResearch#18650: Sanitize malformed tool messages + auto-recover on API 400 - NousResearch#18607: Emergency compression before max_iterations exhaustion - NousResearch#18603: Compression fallback to main model on 413 rate limit - NousResearch#18638: Pass threshold_percent on model switch - NousResearch#18663: Strip extra_content from tool_calls for strict APIs - NousResearch#18618: Forward explicit_api_key to OpenRouter - NousResearch#18632: Show cache tokens in /insights breakdown - NousResearch#18614: Add idempotency guard for patch duplicate loops - NousResearch#18600: Raise ValueError when HERMES_HOME unset in profile mode - NousResearch#18616: Allow ZWJ emoji in context files - NousResearch#18582: Reload .env on /restart - NousResearch#18547: Stabilize system prompt prefix for KV cache reuse - NousResearch#18692: Strip FTS5 operators from session search truncation terms Fix: Add order_by_last_active=True to list_sessions_rich call (pre-existing commit 142b4bf code sync)
Contributor
|
This looks implemented on current Evidence:
Thanks for the original fix; the later redesign removed the code path it targeted while preserving the intended behavior. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
_truncate_around_matches()treats FTS5 boolean operators (AND,OR,NOT) as content terms when splitting the query. Since common English words like "and" appear in virtually every conversation, this produces false match positions and mis-centered truncation windows.Similarly, FTS5 syntax like
NEAR(...), column filters (role:user), and special characters (+,*,^) pollute the search terms.Related Issue
Fixes #4238
Type of Change
Changes Made
How to Test
pytest tests/ -q— all tests should passChecklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture and workflows — or N/A