RAG: spike — chunking + search strategy

[qiuethan]

Blocked by

No response

What

A timeboxed spike that settles two decisions the indexing pipeline is built on top of: chunk size and overlap for the document shapes we actually have, and whether hybrid search (keyword + vector) is worth it at our corpus size or pure vector is enough to start.

Why

Both questions sat in the RAG epic's "Open questions" while the pipeline sub-issues were written as though they were answered. Whoever picks up the chunking pipeline (#175) has to decide them anyway — either deliberately, or by accident in the first commit that hardcodes a number. Deciding by accident is the expensive version: chunk size is not a parameter you change casually once a corpus is embedded, because changing it means re-embedding everything.

The output is a decision, not a deployment. Nobody is worse off today without this shipping; whoever builds #175 is worse off without it being decided.

Area

service, docs-system

Done looks like

• A recommended chunk size and overlap, with the reasoning, written into services/documentation-system/docs/ARCHITECTURE.md.
• A hybrid-vs-pure-vector recommendation for our corpus size, with the reasoning, written into the same place.
• Both carried into RAG: chunking + indexing pipeline (embed, upsert, re-index, delete) #175's design notes, so whoever builds the pipeline reads them without going looking.
• Both tested against real catalogued content, not synthetic text — Google Docs ingested through the existing connector.
• Any throwaway harness is either deleted or promoted deliberately into the evaluation set (RAG: retrieval quality evaluation set #178) rather than left in the tree.

Design notes

Google Docs, Notion pages and GitHub READMEs have materially different shapes — a README is short and structured, a meeting-notes Doc is long and loosely structured. A single chunk size may not serve all three, and "one size, chosen badly" is the outcome this spike exists to avoid.

Only the Google connector is built today, so the spike can only test one shape for real. Say so in the recommendation rather than over-claiming it generalizes.

Out of scope

• Shipping the pipeline itself — that is RAG: chunking + indexing pipeline (embed, upsert, re-index, delete) #175.
• Choosing the embedding model, which RAG: /embed endpoint on the LLM service (Bedrock-backed) #173 settles.
• Building the evaluation harness — RAG: retrieval quality evaluation set #178 owns that, though this spike may prototype something it later absorbs.
• Recording the decision anywhere it can rot. A choice that lives only in a comment thread is lost by the time someone needs it, which is the exact failure this platform exists to fix.