fix(graph): answer a graph that is out of date instead of refusing it, and publish one whose only fault is a file the policy skipped - #189
Merged
Conversation
…d manifest The build manifest folds six engine-identity inputs and one config-content hash into a single value, so a reader can only ask whether the fold changed, never which class of input moved. Report the inputs alongside the hash and add the reconstruction test that answers the narrower question. Identity is proven by re-folding the current inputs with a store's recorded config hash rather than by comparing a stored identity field, so stores written before this check can still be classified, and corpus policy — which no snapshot records — cannot pass as config drift.
…ding Freshness inspection produced an exact binding token only for a store it could call fresh, so every reason to stop calling it fresh collapsed into the same refusal. Classify one of those reasons separately: a store that would read fresh if its config content had not drifted is still an exact description of the source it indexed, and is now bound and offered under its own observation. The token is produced only when engine identity reproduces, the indexed corpus, branch, corpus digest and grammar all still match, parse health is clean, and every inspection completed. Anything unproven still refuses. Freshness validation now separates its two comparisons. Whether the two observations agree with each other stays a race check for both classes; whether they agree with the stored snapshot is the freshness question the caller already answered, and for a config-drifted read it is the condition being served rather than a race.
The read handshake adopted a session only for a fresh observation. It can now also adopt the config-drifted one, behind an explicit option, and reports which class it bound so the caller can label what it emits. Consumers that cannot label a degraded answer keep the previous behaviour by not asking. Final revalidation is bound to the class the session opened under. A store that changes class mid-read carries a label its buffered records no longer earn, so that response is discarded rather than relabelled at output.
query, get and impact refused outright when a build input drifted, so one dependency bump made structural retrieval unavailable until a full rebuild. They now serve the drifted store and declare it. The declaration is a record, not an exception, and carries the same recovery command the refusal did. What it labels is deliberately narrow: definitions, containment and source bytes do not depend on compiler configuration, and the returned source is already proven byte-identical to what was indexed, so they are returned unlabelled. Resolution does depend on it, so every fact reached by following an edge — callers, call relations, unresolved references — is marked stale. A fresh response is unchanged: every added field appears only when the store is drifted.
…vocabulary Scope compared one manifest hash and threw its own rebuild-required error, so the same drifted dependency that made the targeted commands report an unavailable graph made scope demand a rebuild instead. One condition had two mechanisms and two error codes. It now classifies through the shared predicate and refuses through the shared record, and answers a config-drifted store labelled, marking flows — chains of resolved edges — rather than the source and definitions it also returns. Scope keeps its own per-file staleness pass rather than adopting the exact freshness handshake. That pass is what lets it answer while source files are being edited, by discarding a moved file's graph facts and re-admitting it as text-only evidence. Binding scope to exact freshness would trade that for a refusal on the first edited file, which is a worse answer, not a safer one.
The integration case pinned the old policy: a tsconfig moduleResolution change with no source edit had to refuse retrieval outright. It now asserts what that change actually costs — resolved edges become unreliable, so the answer is served and labelled, while re-staging on the next sync is unchanged.
Describe what separates engine identity from config content, which half of a degraded answer stays trustworthy, and why scope keeps a different tolerance for drifted source than the targeted commands do.
Serving from a store the graph declines to call fresh is exactly the state where an implicit repair or checkpoint would be tempting. Assert the whole .mex directory is unchanged after a drifted query and scope.
A store with both a drifted dependency and an edited source file was refused with the configuration change as its reason and `mex graph refresh` as its remedy. Configuration is the input the gate now excuses, so it can never be why a read was refused; reporting it sends the reader to revert an edit that was never the blocker. Freshness inspection reports whether config drifted under an engine identity that still reproduces, and refusals rank the config-class diagnostics last when it did, falling back to them only when nothing else objected. Also fixes the scope flow label, which was being attached to the planning envelope that carries a record rather than to the record itself, so flows were never marked stale.
… skipped Skipping an oversized file instead of aborting the build made the candidate validate as degraded, and the publish gate admitted only two degraded diagnostic codes. The code the skip path emits was never added to that list, so the gate discarded the exact candidate the skip path exists to produce: a long build, no published graph, and a status of missing. A file the corpus policy declined, and the bounded notice that more were declined than were listed, are gaps the build made deliberately and would reproduce exactly. Corpus-wide breaches and incomplete inspections stay blocking, because those mean the observation itself is untrustworthy rather than that one file is missing from a trustworthy one.
…tence A failed publication printed the status it refused and discarded the diagnostics the error already carries, so a long build that produced no graph gave no way to learn which file was responsible, what was skipped, or what blocked it. The graph commands now render those diagnostics, bounded, with their paths, any remediation command, and any retained recovery path. Exit status was already non-zero on this path and is unchanged; a shell pipeline reporting zero is reading the exit code of the last command in the pipe rather than of mex.
…y so Any partially parsed file made freshness inspection report degraded, and the read gate accepted only fresh, so one file the parser could not finish cost a repository every structural read. This scaffold's own index trips it. Bindable shortfalls are now a set rather than one class, and an unfinished parse joins config drift in it. The two make different claims and are reported differently: drifted configuration means resolved references may be out of date, so edge-derived records stay marked stale, while an unfinished parse means the store is merely incomplete — every fact in it is still true, there are fewer of them than the repository contains, and the response says how many files are affected and which failed. Everything else still refuses: a changed corpus, branch, digest, grammar or engine identity, an unfinished inspection, or a demanded rebuild.
…he read Editing one indexed file refused every targeted read, which is the state an active repository is in most of the time — the reported symptom was a graph that answered for hours after a rebuild and not for days. Scope already survives this by dropping a moved file's graph facts and re-admitting it as text-only evidence. A command returning exact node coordinates has no such fallback, so it does the equivalent: the complete set of drifted paths is excluded, the rest of the repository answers, and the response names every file it left out so a caller can see the answer is partial rather than inferring it from a short result. Completeness of that set is what makes this safe, so it is bound to the change-path ceiling: a truncated change list cannot be exhaustively excluded from, and a store past that ceiling refuses as before. A node whose own file drifted is reported as excluded rather than missing, and a target that resolves only into excluded files says so instead of returning nothing.
… bytes The manifest hashed the raw content of every package.json, tsconfig and jsconfig in the repository, so a dependency version, an npm script, a description field or a reindent was indistinguishable from a change to how modules resolve. Each of them invalidated the whole index, which in an active repository is close to permanent invalidation. Config inputs are now projected down to the fields that decide what the compiler resolves and which files it reads — extends, references, include, exclude, files, and paths, baseUrl, moduleResolution, module, target, jsx, allowJs and checkJs from compilerOptions; type, workspaces, imports, exports and dependency names from a package. tsconfig is parsed as JSON with comments rather than compared as text, and object keys are ordered canonically so reformatting cannot register as a change. The hazard here runs the other way from over-invalidation: a field that affects extraction and is missing from the projection would let a stale index read as current with nothing to say otherwise. Every included field has a test asserting it still invalidates, and anything unparseable, unrecognized or malformed falls back to its exact bytes. Existing stores read as config-drifted once after this change, which the read path now answers and labels, and a rebuild clears.
Staging re-resolves each source path after reading it and required the result to equal the path the file was opened by. On Windows those two strings can differ in case alone: realpathSync preserves the casing it is handed, and a path that reaches the graph through the TypeScript compiler host arrives lowercased. One file in a public 381-file TypeScript repository resolved that way, its device, inode, size and timestamps all matched, and the byte comparison rejected it — failing the entire build, one file at a time. Path comparison is now case-insensitive where the volume is, and exact everywhere else, so a case-sensitive filesystem cannot have two different files accepted as one. The file identity checks around it are untouched: those are what actually detect a path repointed at different content, and a name comparison was never doing that job.
…identity Describe the three bounded shortfalls a bound read tolerates and how each is reported, the matching judgement on the publish side, why config inputs are identified by field rather than by byte, and the path-comparison trap that made one file fail a whole build.
The degraded-read fixtures bumped a dependency version, which no longer identifies a different build now that config inputs are projected onto the fields that affect extraction. They change a package type instead, which is the property under test.
Yashasvi2229
marked this pull request as draft
September 9, 2026 14:30
Yashasvi2229
marked this pull request as ready for review
September 9, 2026 14:52
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two gates made the same mistake in opposite directions:
degradedwas treated asunusable.The read gate
The build manifest reports the inputs it folds. Six of the seven are engine identity — schema, compiler, extractor and resolver versions, grammar hash, corpus policy. The seventh is config file content.
graphManifestnow returns those inputs alongside the hash, plus a puregraphManifestHash().A bindable store may fall short of
freshin three bounded ways, each reported separately because each makes a different claim:config-drift— compiler inputs moved, so resolved references may be out of date. Proven by reconstruction: re-fold the current inputs with the store's own recordedconfigHashand require its recordedmanifestHashback. No schema change, and existing stores are classifiable without a rebuild.parse-degraded— some files parsed partially, so the store is incomplete. Every fact in it is still true; there are fewer of them.source-drift— some indexed files changed, so what the store says about those files describes an older revision.query/get/impact/scopeanswer such a store and say so. The response opens with astatusrecord naming the reasons, listing what was left out, and carrying the recovery command as data rather than as an exception.Drifted source is answered around, not refused. The complete set of drifted paths is excluded and the rest of the repository answers. A node whose own file drifted is reported as excluded rather than missing (
NODE_SOURCE_DRIFTED/TARGET_SOURCE_DRIFTED). Completeness of that set is the safety property, so it is bound to the change-path ceiling: a truncated change list refuses as before.scopeuses the same classifier and refusal record, instead of its ownGRAPH_REBUILD_REQUIREDfor a condition the other commands reported asGRAPH_UNAVAILABLE. Its wider per-file tolerance is kept deliberately — it re-admits moved files as text-only evidence, which the coordinate-returning commands cannot do.A refusal names the input that actually blocked it, ranking config-class diagnostics last when config drift was the input the gate excused.
Engine identity still fails closed. A store written by an incompatible schema, compiler, extractor, resolver, grammar or corpus policy refuses every read, unchanged.
The publish gate
A candidate is published when its only fault is a file the corpus policy skipped. fix(graph): survive un-indexable files, surface unresolved references, and stop storing references the resolver already turned into edges #174 stopped an oversized file aborting the build, but the publish gate admitted only two degraded diagnostic codes and the skip path's code was never added — so it discarded the exact candidate that fix produces: a long build, no published graph, status
missing. Corpus-wide breaches (GRAPH_SOURCE_CORPUS_LIMIT_EXCEEDED) and incomplete inspections stay blocking.A failed maintenance run explains itself, rendering the diagnostics the error already carried — which file, what was skipped, what blocked it — instead of one sentence naming the status it refused.
Two defects found while verifying
Config inputs are identified by what they change, not by their bytes. Reindenting a
package.jsonor adding adescriptionused to invalidate every graph read. Only fields that affect extraction are hashed.A re-resolved path is compared as a name, not as bytes.
realpathSyncpreserves the casing it is handed while the compiler host lowercases, soC:\Users\…was compared againstc:\users\…and a containment guard rejected the file. Deterministic, one file at a time, and it costhonoits entire index. Fixed across all five re-resolution sites.Why
Addresses #140 observation 4 — an overbroad staleness gate that made graph reads "unavailable most of the time" in an active repository. Any byte change to any
package.jsonortsconfiglockedquery,get,impactandscopebehind a full rebuild.Honest limits
mex graphremains the way to make a graph current.scopeand the coordinate-returning commands keep different tolerances by design. The classifier and refusal vocabulary are unified; the gates are not merged.Type of change
How to test
The read gate
package.json.mex graph query who-calls <symbol>.Before:
GRAPH_REBUILD_REQUIRED(scope) /GRAPH_UNAVAILABLE(query). After: astatusrecord with"reasons":["config-drift"]and results carrying"stale": true, then the answer."reasons":["config-drift","source-drift"]and an answer that excludes the drifted file and says so.where-definedis not labelledstale— no resolution produced it.manifestHashto something unreproducible and verify every command returnsGRAPH_UNAVAILABLErather than a labelled answer.The publish gate
maxSourceFileBytes.Before: a long build,
The isolated graph candidate validated as degraded, no graph, statusmissing. After: the graph publishes, and the skipped file is reported with its size, the limit, and thegraph.ignoreremediation.Path comparison
hono. Before:Could not stage 1 source file(s): src/router/reg-exp-router/prepared-router.ts, exit 1, no graph. After: 381 files, 13,240 nodes, 29,545 edges, fresh.Verified locally on: a synthetic workspace, a 215-file corpus, a 655-file / 18-config corpus,
hono(381 files, previously unindexable), and a 3,253-file / 120-config monorepo (previously unpublishable — now 53,069 nodes / 86,232 edges) which is the shape #140 reported.Checklist
npm test) — see note belowNote on the suite. 61 failures on this branch against 63 on clean
main, same machine. No graph-side failure is unique to the branch. Three branch-only names are a varying subset ofteam-inbox-spec-authoring-real.contract.test.ts, which fails 11/40 in isolation on cleanmainand 10/40 here, overteam/inbox/wikicode this branch does not touch.npm run evalfails identically on cleanmainwith aRangeErrorreadinglsh_buckets— v3 band hashes exceedNumber.MAX_SAFE_INTEGERandnode:sqliterefuses to return them as numbers. Pre-existing and unrelated;npm run eval:graph -- --validatepasses.No schema change.
DB_SCHEMA_VERSIONis unchanged and no rebuild is required. Existing stores are classifiable as-is, because identity is proven by reconstruction rather than by a newly stored field.Code-graph changes
mainLanguageExtractororFrameworkResolverinterface — n/a, neither is touchedcore / discuss-firstissue is linked above