fix(search): point a match at where it actually is - #368
Open
fstubner wants to merge 1 commit into
Open
Conversation
A search result rendered `Match ${message_start_index}-${message_end_index}`
beside a tool whose parameter is documented as "Message offset for
pagination". An agent auditing this project read that as an invitation and
followed it, which is the failure: those are not offsets.
`getSessionDetail` pages by POSITION in a session's timestamp ordering. A
retrieval unit stores the `message_index` VALUES at its edges. The two agree
only while numbering is dense and monotonic in time order, and real
transcripts are neither — on a live index 430 of 9,728 windows (4.4%) printed
a range that read backwards, `Match 5987-2108`, because one session carries
828 duplicate messages and 862 places where index order disagrees with time
order. Following one landed three weeks from the match.
Matches now carry `detail_offset`, resolved on read, and the line says what to
do with it: `- Match (xtctx_session_detail offset=1274): ...`.
Resolved on read rather than stored because the stored edges are part of a
window's id: changing them orphans every vector in every existing index and
costs each user a full re-embed, measured at 92 minutes on the project that
exposed this. At most three matches per session over five sessions, each an
indexed count, is the cheaper side of that trade. A lookup that fails leaves
the pointer off and returns the match anyway — a signpost is an aid, not the
answer.
Verified against the index that exposed it: 75 of 75 matches now land on their
window's own first message, checked through the public detail path. The
regression test builds a session whose indices deliberately do not ascend with
time — the shape a `/compact` re-ingestion produces — so position and index
cannot be mistaken for each other; reporting the raw index again turns both
assertions red.
The three orderings that must agree are now named in each other's comments:
`selectSessionMessages`, `getSessionDetail`, and the new
`messageOffsetInSession`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A search result rendered
Match ${message_start_index}-${message_end_index}beside a tool whose parameter is documented as "Message offset for pagination". An agent auditing this project read that as an invitation and followed it. Those are not offsets.getSessionDetailpages by position in a session's timestamp ordering. A retrieval unit stores themessage_indexvalues at its edges. The two agree only while numbering is dense and monotonic in time order — real transcripts are neither.Measured on a live index:
Match 5987-2108was a real rendered line. Following it landed three weeks from the match.The fix
Matches carry
detail_offset, resolved on read, and the line now says what to do with it:Resolved on read rather than stored, because the stored edges are part of a window's id — changing them orphans every vector in every existing index and costs each user a full re-embed (measured: 92 minutes on the project that exposed this). At most three matches per session over five sessions, each an indexed count, is the cheaper side of that trade. A failed lookup omits the pointer and still returns the match; a signpost is an aid, not the answer.
Verification
Against the real index that exposed it: 75 of 75 matches now land on their window's own first message, checked through the public detail path.
The regression test builds a session whose indices deliberately don't ascend with time — the shape a
/compactre-ingestion produces — so position and index can't be mistaken for one another.message_start_indexagain708 tests pass, typecheck and lint clean.
The three orderings that must stay identical (
selectSessionMessages,getSessionDetail,messageOffsetInSession) now name each other in their comments.