graph query/get/impact run a full store-wide integrity audit twice per read -- 143s for one lookup on a 288 MB store, vs 6.9s for graph scope

[Yashasvi2229]

Summary

Every targeted graph read — graph query, graph get, impact — runs a full store-wide integrity audit twice before it answers. On 0.8.0 the audit, not the answer, is essentially the whole cost: a single-symbol where-defined on a 3,227-file store takes 135 s and impact takes 143 s, while graph scope — a broader, source-backed retrieval over the same store — answers in 6.9 s.

The second audit is deliberate and correct (src/graph/read-session.ts:441-445): a store that changed class mid-read would carry a label its buffered records no longer earn. The defect is not that the check runs twice. It is that one check costs far more than answering the question, and scales with store size rather than with the question.

The shipped agent instructions tell agents the opposite: templates/AGENTS.md says these commands are "exact and cheap. This is the strongest part of the graph."

Measurements

mex-agent 0.8.0, graph schema v4, Node v24.11.0, Windows 11. Each store built with mex graph rebuild into a scratch copy of the corpus. 3-5 runs per command, wall clock, cold process each time.

Corpus A — 215 files, 8,591 nodes, 20,721 edges, 49 MB store

command	median	vs scope
mex --version (process floor)	1.9 s	—
graph scope "<task>"	4.1 s	1.0x
graph status	8.6 s	2.1x
graph query who-calls <symbol>	14.5 s	3.5x
graph get <id>	14.6 s	3.6x
impact <symbol>	14.5 s	3.5x

hono (public OSS) — 381 files, 13,240 nodes, 72 MB store

command	median
graph scope "route matching"	3.8 s
graph status	10.6 s
graph query where-defined Hono	18.7 s
impact Hono	18.7 s

Corpus B — 3,227 files, 288 MB store

command	runs	median
graph scope "<task>"	6.9 / 6.9 / 6.9 s	6.9 s
graph status	54.6 / 54.8 / 54.3 s	54.6 s
graph query where-defined <class>	116.2 / 136.8 / 134.7 s	134.7 s
impact <class>	140.0 / 143.1 / 143.3 s	143.1 s

scope grows 4.1 s -> 6.9 s across a 15x file-count range. query grows 14.5 s -> 134.7 s over the same range. The two commands read the same store.

1. The cost is a store-wide integrity audit, not source hashing

inspectGraphStatusWithFreshObservation (src/graph/status.ts:404) runs, per call:

• PRAGMA quick_check(100) (status.ts:2420)
• 19 whole-table LEFT JOIN ... COUNT(*) invariant queries (inspectCoreInvariants, status.ts:2104)
• a full ordered cursor walk of node_fingerprints and lsh_buckets, recomputing bandHashInts for every fingerprint and comparing all 32 band hashes (inspectFingerprintInvariants, status.ts:2236)
• inspectLiveSources (status.ts:1806): read and SHA-256 every supported source file
• one git status --porcelain=v2

--cpu-prof on graph status, corpus A (9.7 s sampled), top self-time:

  2418 ms  sqlite Statement.get
  1308 ms  inspectFingerprintInvariants
   825 ms  (idle)
   725 ms  lstat
   661 ms  sha256 (digest + Hash + update)
   270 ms  sqlite Statement.all

Timing the phases directly against corpus A's 49 MB store (node:sqlite, read-only):

   251 ms  PRAGMA quick_check(100)
   887 ms  invariant: LSH bucket(s) without a fingerprint
  1346 ms  invariant: LSH bucket(s) without a node
  2282 ms  = 8 of the 19 core-invariant queries
   828 ms  iterate lsh_buckets ORDER BY ref, band, band_hash   (261,920 rows)
    36 ms  iterate node_fingerprints ORDER BY ref              (8,185 rows)

lsh_buckets holds 32 rows per fingerprint — 261,920 rows for 8,591 nodes on a 215-file repo. Two invariant queries join that table against node_fingerprints and nodes, and the fingerprint walk then re-derives every band hash in JS. The source-hashing pass the audit is nominally about is ~0.66 s of a ~6.7 s inspection.

The same phases on corpus B's 288 MB store (53,069 nodes, 39,931 fingerprints, 1,277,792 lsh_buckets rows):

  1209 ms  PRAGMA quick_check(100)
  9500 ms  invariant: LSH bucket(s) without a fingerprint
 17557 ms  invariant: LSH bucket(s) without a node
 27406 ms  = 8 of the 19 core-invariant queries
  5941 ms  iterate lsh_buckets ORDER BY ref, band, band_hash
   276 ms  iterate node_fingerprints ORDER BY ref

Two queries account for 27.1 s of the 54.6 s graph status. They are also superlinear: lsh_buckets grows 4.9x from corpus A to corpus B (261,920 -> 1,277,792 rows) while LSH bucket(s) without a node grows 13x (1.35 s -> 17.6 s).

The independent confirmation is graph scope: it opens the same store, hashes every indexed file's live source (runScopeAgentSession, cli-agent.ts:1968), answers, and returns in 4.1 s. Corpus hashing is not what makes an inspection expensive.

2. The targeted read path pays for the audit twice

graph query / graph get / impact route through runFreshAgentSession (cli-agent.ts:1936) -> loadFreshGraphReadSession (read-session.ts:346), which calls inspectObservation once up front and again in revalidateFreshness (read-session.ts:438-440) after the task has buffered its output. graph scope uses the stable path and inspects zero times.

--cpu-prof on graph query (18.2 s sampled) shows every inspection frame exactly twice, with matching self time:

  2425 ms  sqlite Statement.get
  2331 ms  sqlite Statement.get
  1221 ms  inspectFingerprintInvariants
  1091 ms  inspectFingerprintInvariants
   363 ms  sha256 digest
   317 ms  sha256 digest
   254 ms  sqlite Statement.all
   249 ms  sqlite Statement.all

The arithmetic closes: query ~= process floor (1.9 s) + 2 x inspection (2 x ~6.7 s) ~= 15.3 s against 14.5 s measured. Answering the actual question is under a second.

The second inspection is not the bug. The comment at read-session.ts:441-445 is right: output is committed under the class it was labelled with, and a store that changed class mid-read carries a label its buffered records no longer earn. Removing the revalidation would trade a slow correct answer for a fast wrong label. The fix has to be to make an inspection cheap.

3. Pre-existing, not introduced by #189

main already contains the #189 merge (bc2d40a). Compared against faa6416 (main immediately before that merge), same corpus content, each store rebuilt with its own code:

	faa6416 (pre-#189)	bc2d40a (post-#189)
graph status	8.5 / 8.7 / 9.4 s	8.4 / 8.6 / 8.8 s
graph query	14.8 / 14.8 / 15.1 s	14.2 / 14.5 / 14.3 s

No difference. Cross-checked by reading both stores with post-#189 code (14.2-15.1 s vs 14.5-14.6 s) to rule out a store-content effect.

inspectCoreInvariants and inspectFingerprintInvariants entered in 790f5c8 (2026-08-22, "gate consumers on verified freshness"); the second inspection in fd38737 (2026-08-23, "bind targeted reads to exact freshness"). Both ship in v0.8.0.

4. The shipped agent instructions say the opposite

templates/AGENTS.md, ## Code Graph:

If you know the symbol name, go straight to it: mex graph query <who-calls|what-calls|where-defined> <symbol> and mex graph get <id> are exact and cheap. This is the strongest part of the graph.
Exploring an unfamiliar task? mex graph scope "<task>" returns bounded, source-backed JSONL context ...

Agents are steered to the 14.5 s / 134.7 s commands as the cheap first move, and to the 4.1 s / 6.9 s command as the exploratory fallback. On a large repo an agent following these instructions literally — impact before an edit, then query, then get <id> to expand — spends six to seven minutes on three lookups.

This belongs in this issue rather than a separate one: the sentence is not independently wrong, it is wrong because of the defect above, and fixing the read path makes it true again with no doc change. It is worth calling out only in that it is the one part that can be corrected immediately and independently, while the perf work is scoped.

Fix direction

Not sufficient as the primary fix: stat-prefilter the source hashing. files already stores size and modified_at alongside content_hash (src/graph/schema.sql:107-118) — the schema comment describes exactly this two-stage filter for incremental re-extraction — so the data is there and inspectLiveSources could use it. But it targets ~0.66 s of a ~6.7 s inspection, and scope's near-flat scaling (4.1 s -> 6.9 s across 15x the files) shows corpus hashing is not the term that explodes. Worth doing; it does not fix this.

What needs to change is that a read runs a maintenance-grade audit at all:

1. Move inspectCoreInvariants / inspectFingerprintInvariants off the ordinary read path. They validate persisted shape — the concern of graph rebuild, graph repair, and a deliberately invoked graph status, not of answering who-calls. graph scope already reads the store without them. Freshness has to prove the indexed source still matches disk and the snapshot identity is unchanged; it does not have to re-prove the store's LSH bands were written correctly, which cannot change while the file is held open immutable.

2. If a structural check must stay on the read path, make it O(question), not O(store). The two LSH joins plus the ordered lsh_buckets walk are the bulk of the cost on both corpora, and they grow superlinearly with node count. A cheap store-identity token (schema version + snapshot hash + database file identity — all already read) proves "this is the store the last audit passed" without re-auditing it. Failing that, the two LSH invariant joins are the single highest-value target: 27.1 s of a 54.6 s inspection on corpus B.

3. Then the second inspection becomes affordable and can stay, which is the outcome the read-session.ts:441-445 comment wants.

Expected shape after (1): targeted reads land near scope — single-digit seconds on corpus A, around 10 s on the 3,227-file store — instead of 14.5 s and 134.7 s.

Repro

# any TS/JS repo; copy the source out first — never build into a repo's own .mex/
mex graph rebuild --root .
time mex graph scope "some task"               # stable path, no inspection
time mex graph status                          # one inspection
time mex graph query where-defined <Symbol>    # two inspections

Ratios hold on any store and widen with node count. --cpu-prof on the third command shows each inspection frame twice.

Environment

• mex-agent 0.8.0 (bc2d40a), graph schema v4, extractor typescript-5.9-v2+tree-sitter-v2
• Node v24.11.0, Windows 11 (26200), NVMe, machine otherwise idle, one measurement at a time
• Corpora: a 215-file TS repo, hono (381 files), a 3,227-file TS monorepo