Skip to content

RAG: re-embed on embedding model change #209

Description

@qiuethan

Blocked by

#174, #175

What

Handle a change to the embedding model or its dimensionality: detect the mismatch, re-embed the affected chunks, and make the cost and duration of a full re-index a known quantity before anyone triggers one.

Why

#174 records the embedding model and dimensionality per row specifically "so a model change is detectable rather than silently corrupting similarity" — but nothing on the board acts on that detection. As written, the vector store can tell that a row was embedded by a different model and has no path to do anything about it.

The failure mode is quiet, which is what makes it worth its own issue. Mixed-model vectors do not error; they rank badly. Similarity scores stay plausible while retrieval quietly gets worse, and the first symptom is the helper bot giving unhelpful answers that look like a prompt problem.

Re-embedding is also a Bedrock spend event. An org with a student budget should know what a full re-index costs before running one, not after.

Area

service, docs-system

Done looks like

  • A model or dimensionality mismatch on read is detected and surfaced, not silently ranked.
  • An operator can re-embed affected chunks without dropping and re-ingesting the corpus.
  • Re-embedding is resumable — an interrupted run does not require starting over or leave the store half-migrated.
  • The cost and rough duration of a full re-index are documented from a real measured run, not estimated.

Design notes

Blocked by both #174 (the schema that records model and dimensionality) and #175 (the admin re-index endpoint). This is a caller of that machinery, not a parallel path — it must not introduce a second entry point of its own.

Worth deciding whether a mismatched row is excluded from results until re-embedded, or ranked anyway. Excluding is the honest default: a stale vector produces a confidently wrong ranking, and returning nothing is easier to notice than returning something subtly bad.

Out of scope

  • Choosing which embedding model to move to. This issue is the mechanism, not the decision.
  • Automatic re-embedding on deploy. An operator triggers this deliberately; a migration that silently spends money on every deploy is a worse failure than the one it fixes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/docs-systemdocumentation-system service & content pipelinearea/serviceBackend API services (generic/cross-cutting)type/featureNew capability or enhancement

    Type

    No type

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions