[ISSUE-844] fix(ai): invalidate an embedding vector when its text changes

[E2ern1ty]

What changes were proposed in this pull request?

Fixes #844.

EmbeddingIndexStore keyed each persisted vector by the entity key, a prefix plus the id and the label, with nothing derived from the value that was embedded. initStore read the presence of that key as "this entity is indexed", so an entity that kept its id and changed its value silently kept the vector of the value it no longer had. Retrieval for the old value still reached the entity while its current value was unreachable, and because the subgraph handed on is verbalized from the current graph, a query about the old value came back with a context showing the new one.

A record now says which text produced it. EmbeddingResult gains a contentHash field, SHA-256 over the entity's chunk list. On rebuild a record is loaded only when that fingerprint matches what the entity would be embedded from now; a mismatch counts as not indexed, so the entity is embedded again. Both paths derive the chunk list through one method, so the text hashed on the way in is the text hashed on the way out. The number of records dropped is logged, since the complaint in the issue is not only that the vector was stale but that nothing said so.

Existing index files are treated as the first of the two options the issue put up for decision. A record without a fingerprint cannot be shown to match the current value, and trusting it would keep exactly the staleness the check is for, so it is not loaded and its entity is embedded once. The other option, valid until the entity is next touched, never comes due, because nothing touches such a record.

That is the whole of the production change: +106 lines across three files, +63 of them in EmbeddingIndexStore.

What is deliberately left out

The file is still only appended to, which is where the issue puts the line: "Pruning orphan records wants a rewrite of the file rather than an append, which is a separate piece of work from invalidation and may be better done as compaction."

So a superseded record stays in the file and is rejected on each later read, and the records of deleted entities stay too. Two things make that liveable:

• The file grows by one record set per distinct value an entity has held, not per change, and a value that comes back is served from the record already there without a request. There is a test for that.
• The harm the issue names does not wait for compaction. "A later entity reusing the same id would silently adopt the old vector" — it no longer does, because the fingerprint on the vector left behind does not match the new entity's text. There is a test for that too, going through the full sequence: the entity is indexed, removed, and its id taken over.

What remains for compaction is only that the file size does not track the graph.

I had a rewrite-on-load in the first revision of this PR and took it out. It cost a public upgradeIndexFile, a temp-file-and-fsync write path, and the bookkeeping to keep a graph that scanned short from taking the whole file with it — about 220 lines to prune history the issue had already scoped out.

One thing to look at: a test fixture is removed

GraphMemoryTest no longer loads an embedding store from src/test/resources/index/LDBCEmbeddingIndexStore, and that 4.6MB file is deleted. Two findings led there, both checkable before the deletion:

• It could not be carried over. Its records line up with the text the current code would embed for 14 of its 163 entities; the other 149 disagree on chunk count, in a pattern consistent with the file having been written when the values were joined before splitting rather than split per value. So there was no honest fingerprint to stamp on it, and no way to embed it again offline.
• It could not have affected a single assertion. Every query in that test embeds to double[0] through MockChatRobot, and EmbeddingVector.match returns 0.0 on a length mismatch, below the 0.5 threshold. Everything asserted there comes from the keyword path, and those 22 assertions pass unchanged. Runtime for that test drops from 27.6s to 0.6s, the 27s having been model retries against a null URL.

If you would rather keep the file, I can drop only the wiring, though it would then be unreferenced. The **/resources/index/** rat exclusion in the root pom is now unused; I left it alone as it is a generic pattern, and can remove it on request.

Two known limits, neither introduced here

• The fingerprint covers the text only, so changing the embedding model or its dimension leaves every record loaded, and on a dimension change recall would quietly go to zero. Same class of staleness, but which properties of a model identify it is a separate decision. Happy to open a follow-up.
• A record set short of a chunk still matches on every record it has, so an entity can be held with part of its vectors — a fingerprint covers all of an entity's chunks at once and carries no ordinal. Reachable through the blank-response skip in indexBatch. This is unchanged from before: a partial set was skipped as indexed then too.

The verify path also takes verbalize(GraphEntity) to be a function of the entity alone, noted in the javadoc where it is relied on. The one implementation in tree is.

How was this PR tested?

• Tests have Added for the changes
• Production environment verified

EmbeddingIndexInvalidationTest drives the store against a local /v1/embeddings endpoint answering with one hot vectors, one dimension per distinct text, so a stored vector says which text produced it: cosine 1 against its own text and 0 against any other. No key and no network needed. Six cases:

case	what it pins
changed value on a vertex	embedded again, holds the current vector and not the old one, and the record just written is then accepted with the file left byte for byte
changed value on an edge	the same for edges
recall through GraphMemoryServer	the old value reaches nothing, the current value reaches the vertex and reads as that value
id taken over by another entity	the vector left behind is not adopted
value that comes back	served from the record already there, nothing appended
record without a fingerprint	not taken on trust, and the cost is paid once

Each case fails against the behaviour it pins, checked by disabling the fingerprint comparison: three of the six fail outright, the other three by the branch they exercise. The byte-identical assertion on the unchanged run is what would catch a fingerprint computed differently on the two paths.

mvn -pl geaflow-ai clean test -Pjdk8: 12 tests, 0 failures, checkstyle and RAT clean. Nothing in geaflow-ai/src/main constructs this store today, so the added per-entity hashing on load has no production caller in tree.