fix(state): index tool_calls and tool_name in FTS5 for session_search#16866
fix(state): index tool_calls and tool_name in FTS5 for session_search#16866yes999zc wants to merge 1 commit into
Conversation
The FTS5 virtual tables (messages_fts, messages_fts_trigram) previously only indexed the content column via external content mode. Tool calls and tool names stored in the tool_calls (JSON) and tool_name columns were invisible to FTS5 search. Root cause: FTS5 triggers only INSERTed new.content into the index. Changes: - Switch FTS5 tables from external content (content=messages) to inline mode so that trigger-inserted content is both indexed and stored - Update all 6 FTS5 triggers to concatenate content, tool_name, and tool_calls when indexing new messages - Extend the short-CJK LIKE fallback to also search tool_name and tool_calls columns Closes: #16751
Bartok9
left a comment
There was a problem hiding this comment.
Good improvement — indexing tool_calls and tool_name is a clear win for session_search.
One potential issue: the DELETE triggers and the delete-half of the UPDATE triggers still reference only old.content:
INSERT INTO messages_fts(messages_fts, rowid, content) VALUES('delete', old.id, old.content);Since the FTS table is now inline (no content=messages), the delete operation needs the exact same concatenated value that was originally inserted. If the delete content does not match what FTS has stored, FTS5 may silently fail to remove the entry or raise a "database disk image is malformed" error on integrity-check.
The delete lines should use the same COALESCE pattern:
COALESCE(old.content, '') || ' ' || COALESCE(old.tool_name, '') || ' ' || COALESCE(old.tool_calls, '')This applies to all 4 delete operations (2 in FTS_SQL, 2 in FTS_TRIGRAM_SQL).
|
Merged via #16914. Your commit Thanks for the diagnosis and direction — indexing
6 regression tests covering INSERT / UPDATE / DELETE + the v10→v11 upgrade path are now in Closes #16751. Appreciate the contribution. |
Problem
FTS5 virtual tables only indexed the
contentcolumn via external content mode. Tool calls (tool_callsJSON) andtool_namewere invisible to FTS5 search, makingsession_searchmiss most tool-call-mediated conversation context.Fixes #16751
Changes
content || tool_name || tool_callstool_nameandtool_callsVerification
All triggers active, snippet() working, DELETE supported, zero data loss on DB rebuild.