Skip to content

M2X-042: record the Phase 2 RAG architecture and what it has not settled - #36

Open
yashpancholi09 wants to merge 1 commit into
feature/m2x-041-gate-preconditionsfrom
feature/m2x-042-rag-design
Open

M2X-042: record the Phase 2 RAG architecture and what it has not settled#36
yashpancholi09 wants to merge 1 commit into
feature/m2x-041-gate-preconditionsfrom
feature/m2x-042-rag-design

Conversation

@yashpancholi09

Copy link
Copy Markdown
Collaborator

The Phase 2 architecture record.

Stacked on feature/m2x-041-gate-preconditions.

The deviation the ticket cannot avoid

The acceptance criterion is "committed before index code". That ordering is no longer available — M2X-043 and M2X-044 were both built and merged first, so every decision the pairing was meant to take was already taken in code. Writing it as a plan would have put the file's first lie in its first paragraph.

So it is a record: decisions as actually taken, each traced to the ticket that took it, plus the questions the built code does not answer.

Both required questions answered

The ticket says the pairing is incomplete without these two.

Why this chunk unit — a segment is the smallest unit carrying a real t_start/t_end, so whole-segment chunks make a chunk's range exact by construction. Fixed-second windows cut mid-sentence; five-minute chapters are several topics and one vector over several topics matches none sharply. Sub-segment splitting was rejected even for oversized segments: an oversized chunk and an approximate citation are both costs, and only one is visibly wrong.

What happens on a low retrieval score — nothing is answered. Four abstention doors, recorded separately because they are not equally good news. Grounding is enforced inside the retry loop, so an invalid citation returns to the model as an error rather than being dropped.

The five OPEN items are the real output

  1. --max-distance 0.48 rests on eight questions, with an answerable band spanning 0.15 of distance and no relation to answer quality.
  2. top_k = 5 has never been varied, and context precision is a direct function of it.
  3. No mm:ss citation has ever been produced against a real meeting — every verified citation to date is a § heading in this repo's own markdown. This is why M2X-045 is written against eval/tiron/.
  4. The RAG prompt default is still latest_version('rag') — the same unpinned-default trap that moved the extraction default from v3 to v5 on a merge.
  5. Nobody has measured the cost side of the 1200-character chunk.

Also states, for M2X-045: ground truth must be segment ids, because a citation resolves to a segment and an expected mm:ss string would score the renderer.

Cross-review: PENDING, not written

The PM half needs two people. One agent authoring both halves produces something shaped like a review with no second opinion in it — the same failure mode as one agent writing the labels, the extractor and the score. Days 1–3 were left pending for the same reason.

Acceptance criteria are therefore partially met, and the doc says so.

…t settled

The ticket asks for this doc "before index code". That ordering is no longer
available -- M2X-043 and M2X-044 were built and merged first, so every decision the
pairing was meant to take was already taken in code. Writing it as a plan would have
put the file's first lie in its first paragraph, so it is written as a record:
decisions as actually taken, each traced to the ticket that took it.

The useful half is the five OPEN items, which are what a pairing would have argued
about and what the built code currently guesses at:

- --max-distance 0.48 rests on eight questions, with the answerable band spanning
  0.15 of distance and no relation to answer quality; M2X-045 re-derives it
- top_k = 5 has never been varied, and context precision is a direct function of it
- no mm:ss citation has ever been produced against a real meeting, because data/ is
  git-ignored and every verified citation to date is a section heading in this repo's
  own markdown -- which is why M2X-045 is written against eval/tiron/
- the RAG prompt default is still latest_version('rag'), the same unpinned-default
  trap that silently moved the extraction default from v3 to v5 on a merge
- nobody has measured the cost side of the 1200-character chunk

Also states, for M2X-045, that ground truth is recorded as segment ids: a citation
resolves to a segment, so an expected mm:ss string would score the renderer rather
than the retrieval.

The cross-review is logged as PENDING rather than written. It needs two people; one
agent authoring both halves produces something shaped like a review with no second
opinion in it. Days 1-3 were left pending for the same reason. M2X-042's acceptance
criteria are therefore partially met, and the doc says so.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants