M2X-042: record the Phase 2 RAG architecture and what it has not settled - #36
Open
yashpancholi09 wants to merge 1 commit into
Open
M2X-042: record the Phase 2 RAG architecture and what it has not settled#36yashpancholi09 wants to merge 1 commit into
yashpancholi09 wants to merge 1 commit into
Conversation
…t settled
The ticket asks for this doc "before index code". That ordering is no longer
available -- M2X-043 and M2X-044 were built and merged first, so every decision the
pairing was meant to take was already taken in code. Writing it as a plan would have
put the file's first lie in its first paragraph, so it is written as a record:
decisions as actually taken, each traced to the ticket that took it.
The useful half is the five OPEN items, which are what a pairing would have argued
about and what the built code currently guesses at:
- --max-distance 0.48 rests on eight questions, with the answerable band spanning
0.15 of distance and no relation to answer quality; M2X-045 re-derives it
- top_k = 5 has never been varied, and context precision is a direct function of it
- no mm:ss citation has ever been produced against a real meeting, because data/ is
git-ignored and every verified citation to date is a section heading in this repo's
own markdown -- which is why M2X-045 is written against eval/tiron/
- the RAG prompt default is still latest_version('rag'), the same unpinned-default
trap that silently moved the extraction default from v3 to v5 on a merge
- nobody has measured the cost side of the 1200-character chunk
Also states, for M2X-045, that ground truth is recorded as segment ids: a citation
resolves to a segment, so an expected mm:ss string would score the renderer rather
than the retrieval.
The cross-review is logged as PENDING rather than written. It needs two people; one
agent authoring both halves produces something shaped like a review with no second
opinion in it. Days 1-3 were left pending for the same reason. M2X-042's acceptance
criteria are therefore partially met, and the doc says so.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Phase 2 architecture record.
Stacked on
feature/m2x-041-gate-preconditions.The deviation the ticket cannot avoid
The acceptance criterion is "committed before index code". That ordering is no longer available — M2X-043 and M2X-044 were both built and merged first, so every decision the pairing was meant to take was already taken in code. Writing it as a plan would have put the file's first lie in its first paragraph.
So it is a record: decisions as actually taken, each traced to the ticket that took it, plus the questions the built code does not answer.
Both required questions answered
The ticket says the pairing is incomplete without these two.
Why this chunk unit — a segment is the smallest unit carrying a real
t_start/t_end, so whole-segment chunks make a chunk's range exact by construction. Fixed-second windows cut mid-sentence; five-minute chapters are several topics and one vector over several topics matches none sharply. Sub-segment splitting was rejected even for oversized segments: an oversized chunk and an approximate citation are both costs, and only one is visibly wrong.What happens on a low retrieval score — nothing is answered. Four abstention doors, recorded separately because they are not equally good news. Grounding is enforced inside the retry loop, so an invalid citation returns to the model as an error rather than being dropped.
The five OPEN items are the real output
--max-distance 0.48rests on eight questions, with an answerable band spanning 0.15 of distance and no relation to answer quality.top_k = 5has never been varied, and context precision is a direct function of it.mm:sscitation has ever been produced against a real meeting — every verified citation to date is a§ headingin this repo's own markdown. This is why M2X-045 is written againsteval/tiron/.latest_version('rag')— the same unpinned-default trap that moved the extraction default from v3 to v5 on a merge.Also states, for M2X-045: ground truth must be segment ids, because a citation resolves to a segment and an expected
mm:ssstring would score the renderer.Cross-review: PENDING, not written
The PM half needs two people. One agent authoring both halves produces something shaped like a review with no second opinion in it — the same failure mode as one agent writing the labels, the extractor and the score. Days 1–3 were left pending for the same reason.
Acceptance criteria are therefore partially met, and the doc says so.