Search assistant replies and checkpoint bodies via the FTS5 index - #385
Merged
Conversation
Searching the session list for a bug or PR number returned nothing when that number only appeared in Copilot's reply rather than in what the user typed. The session store ships an FTS5 `search_index` table covering full turn text and every checkpoint field, but the list filter never touched it. Deep search built a LIKE predicate over `turns.user_message`, checkpoint title and overview, file paths, and refs. Assistant replies, checkpoint history, work done, technical details, and next steps were all unreachable. `SearchSessionsFTS` did use the index, but only the Copilot SDK search tool called it. Deep search now also matches `s.id IN (SELECT session_id FROM search_index WHERE content MATCH ?)` when the index is present. The index clause is additive rather than a replacement. FTS5 tokenizes, so it misses substrings inside a token: searching "1137" finds "bug 1137" but not "PR1137", which LIKE catches. Keeping both makes the result set a strict superset of the old behavior. Stores without the index get an `assistant_response` LIKE clause instead, so assistant text is searchable there too. Query terms are wrapped in double quotes before being passed to MATCH, so FTS5 operators (`-`, `*`, `NEAR`, `^`) are treated as literals and can't turn a search into a syntax error that breaks the whole list. A whitespace-only query skips the MATCH clause rather than issuing an empty phrase. Verified against a real 20k-session store: searching "1137" went from 6 sessions to 13. Tests: assistant-only match, checkpoint body match, no-match, FTS operator characters, grouped list path, and filter-builder shape for both the indexed and fallback branches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 345398e7-e955-4014-af51-893af8317955
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Searching the session list for a bug or PR number returned nothing when that number only appeared in Copilot's reply rather than in what the user typed.
What was wrong
The session store ships an FTS5
search_indextable covering full turn text and every checkpoint field, but the list filter never touched it. Deep search built a LIKE predicate overturns.user_message, checkpoint title and overview, file paths, and refs. Assistant replies, checkpoint history, work done, technical details, and next steps were all unreachable.SearchSessionsFTSdid use the index, but only the Copilot SDK search tool ever called it.The change
Deep search now also matches
s.id IN (SELECT session_id FROM search_index WHERE content MATCH ?)when the index is present.The index clause is additive rather than a replacement. FTS5 tokenizes, so it misses substrings inside a token: searching
1137findsbug 1137but notPR1137, which LIKE catches. Keeping both makes the result set a strict superset of the old behavior.Stores without the index get an
assistant_responseLIKE clause instead, so assistant text is searchable there too.Query terms are wrapped in double quotes before being passed to MATCH, so FTS5 operators (
-,*,NEAR,^) are treated as literals and can't turn a search into a syntax error that breaks the whole list. A whitespace-only query skips the MATCH clause rather than issuing an empty phrase.Verification
Against a real 20k-session store, searching
1137:The recent session that prompted the report (the number appeared only in an assistant reply) is now in the results.
Tests