Fix/FTS5 preflight coverage, store protection, and a discoverable telemetry opt-out - #172
Merged
Yashasvi2229 merged 8 commits intoSep 6, 2026
Conversation
The probe added in #168 guarded only the writable open path. Readers query nodes_fts / source_chunks_fts, so a store built on an FTS5-capable machine and copied to one without it still failed with SQLite's raw 'no such module: fts5' from check, scope, query, get, and impact. Running the probe before the store is opened also means a doomed environment never creates a handle, rather than opening one and closing it on the way out.
The wiki index's wiki_fts table has the same FTS5 dependency as the graph but never probed for it, so `mex wiki rebuild-index` still failed with SQLite's raw 'no such module: fts5'. It now reports WIKI_INDEX_FTS5_UNAVAILABLE, a new code, rather than borrowing WIKI_INDEX_REBUILD_REQUIRED, whose remediation would send the user round a loop rebuilding an index no rebuild can fix. The probe moves to sqlite.ts, beside openSqlite: FTS5 describes the SQLite build rather than the code graph, and the wiki architecture rule lets the wiki reach that module but not database.ts. database.ts re-exports it, so existing callers are unchanged. It also takes an injected opener now, which lets the failure paths be tested without module mocking that could not reach an internal call anyway. Both store writers also ensure .mex/.gitignore exists before creating anything, which previously only mex setup did (#110).
Turning telemetry off has always been one command, but the only switch lived under `mex config set`. Issue #110's reporter typed `mex telemetry disable`, got 'unknown command', read `telemetry --help`, found two commands that only show things, and then guessed at env var names. The env opt-outs were documented solely in the first-run notice, which scrolls past once and never returns. Adds `telemetry disable`/`enable` writing the same ~/.mex/config.json key as `config set`, lists both env opt-outs in the group's help, and has `status` say what to change rather than only naming a reason code. Disable and enable report when an env opt-out or a dev checkout outranks the value just written, so neither ever claims an outcome the next invocation contradicts. No new setting and no change to the gate's precedence.
PR #168's error message told users to 'see COMPATIBILITY.md for which versions are known to work', but that document said nothing about FTS5 — it repeated the same >=22.5 range that had just failed them. COMPATIBILITY.md now has an FTS5 section: what it is needed for, that it is a compile-time property of the Node build rather than the version number, a one-line command to test the Node you actually run, and the two known data points as reports rather than a supported-range claim. The error message points at that section and stops implying a version answer exists. engines stays >=22.5. FTS5 availability does not track version order, so narrowing the range would lock out working builds without excluding broken ones, on one field report. Also records what the v0.6.3 fallback costs: the code graph shipped in 0.7.0, so the release we send Node-constrained users to has none of the feature they would be falling back for (#110).
Asserts the mechanism (the ignore file exists before the store does) and the consequence the reporter actually hit (`git add -A` cannot stage the database), plus that an existing hand-written ignore file keeps its own rules.
The ignore guard sat before the missing-index refusal, so a mistyped `mex graph refresh` in an unindexed checkout created .mex/.gitignore and then errored. Rebuild — the mode that creates a store — still gets the guard before anything is written.
The build path refused early, but the two entry points that open wiki.db directly — contract status inspection and the read session — let SQLite's raw 'no such module: fts5' escape. Reachable without copying anything between machines: build the index on one Node, switch versions with a version manager, read it on another. Status reports degraded rather than corrupt, because nothing is wrong with the store and no rebuild on this Node would improve it. The read session reuses the existing INDEX_UNAVAILABLE code, which already means 'this index cannot be read right now' and is exactly true here, so the frozen read-error vocabulary is unchanged and the actionable detail rides in the message. The probe-to-diagnostic mapping moves into its own index/fts5 module. open.ts is the wrong home once a second layer needs it, and query must not import an opener to ask a question about the engine. publish.ts needs no guard: it only runs after a rebuild that already probed, and reads wiki_meta plus integrity_check, neither of which touches FTS5.
Yashasvi2229
deleted the
fix/fts5-preflight-coverage-and-optout-discoverability
branch
September 6, 2026 18:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Finishes the four-part report in #110. PR #168 fixed the first part —
mex graphnow names the problem when Node's bundled SQLite lacks FTS5 — and this picks up what it scoped out, plus a gap that fix left behind.nodes_fts/source_chunks_fts, and the wiki index has the same dependency viawiki_fts, somex check,graph scope/query/get,impactand everymex wikicommand could still hit SQLite's rawno such module: fts5. The probe moved fromgraph/db/database.tstograph/db/sqlite.ts— it describes the SQLite build, not the code graph, and the wiki architecture rule lets the wiki reach that module but notdatabase.ts.database.tsre-exports it, so existing callers are unchanged..mex/.gitignoreis written by store writers, not only by setup.mex graph rebuild/refresh/repairandmex wiki rebuild-indexnow ensure it exists before creating anything.mex telemetry disable/enable, writing the same~/.mex/config.jsonkey asmex config set telemetry on|off. The env opt-outs are named intelemetry --helpand instatus.enginesstays>=22.5— see Why.Why
Closes #110.
Each remaining item was a way for a correct tool to look broken:
mex wiki rebuild-indexrun on its own reproduced the original confusing failure verbatim.mex graphin a checkout that had never run setup, and their nextgit add -Awould have committed a broken database plus its WAL and SHM files.mex config. The reporter typedmex telemetry disable, gotunknown command, readtelemetry --help, found two commands that only show things, and then guessed at env var names that didn't exist. They said they audited TELEMETRY.md againstsrc/telemetry/index.tsline by line before running anything, because their work touches medical data — that is exactly the user for whom "where is the off switch" must be a ten-second answer.>=22.5range that had just failed them.On not narrowing
engines: FTS5 is a compile-time option that Node neither documents nor guarantees, so availability is a property of the build, not the version number — official installers, distro packages and self-compiled Node can differ at the same version. Setting a floor from one field report would lock out working 22.x/23.x installs while still admitting a broken build above the floor. COMPATIBILITY.md instead carries a one-line command to test the Node you actually run, and the two known data points as reports rather than a supported-range claim. More data points would justify a floor; one does not.Type of change
How to test
FTS5 preflight (behaviour is unchanged on any Node that has FTS5, which is the common case):
node --no-warnings -e "new (require('node:sqlite').DatabaseSync)(':memory:').exec('CREATE VIRTUAL TABLE t USING fts5(x)')" && echo "FTS5 ok"— the check now documented in COMPATIBILITY.md.npx vitest run src/graph/__tests__/database-fts5.test.ts src/wiki/index/__tests__/fts5.test.ts— the probe's failure paths are driven through an injected opener, so they are reachable without an FTS5-less Node. Covers the write path, both graph read paths, and a real, intact wiki index that this engine cannot read.Store protection, in a scratch repo with no
.mex/:git init, add one.tsfile, then runmex graph.git add -A && git status—.mex/.gitignoreand your source are staged;graph.dband its sidecars are not. Onmain, all three databases stage.npx vitest run test/store-ignore-protection.test.tsTelemetry:
mex telemetry --help— listsdisable,DO_NOT_TRACK=1andMEX_TELEMETRY=0.mex telemetry disable, thenmex telemetry status— the reason is named and explained.DO_NOT_TRACK=1 mex telemetry enablereports that the env var still outranks what was just written.npx vitest run test/telemetry-optout-cli.test.tsChecklist
npm test)On "tests pass": measured as branch-vs-
main, not as a green board. The full suite was run on this branch and again in a worktree atmain, and the failure sets diffed by name: 69 failures here, 71 onmain, with one test failing only here and two only onmain. All three pass in isolation — that is parallelism flake, not a regression. The ~70 baseline failures are pre-existing on this Windows machine (the documentedfindConfigtrap:TEMPsits under a home directory that is itself a git repo, so tests assertingbranch: nullseebranch: "main").npm testalso does not run the evaluator tests. CI is the arbiter.Not breaking, and why each near-miss isn't:
assertFts5Availablemoved fromgraph/db/database.tstograph/db/sqlite.tsand is re-exported from its old home, so any existing import keeps working. It gained an optional injected-opener parameter; the default is the previous behaviour.WIKI_INDEX_FTS5_UNAVAILABLEis a new wiki diagnostic code — additive, and registered with the coverage test's emitter. The frozenWikiContractReadErrorvocabulary is untouched: the read session reusesINDEX_UNAVAILABLE, which already means "this index cannot be read right now".Tested locally with a real project: two private repositories (111 and 64 indexed source files), with the store redirected to a temp path so neither repository's own
.mex/was opened or written. Both produced node counts identical to their existing live stores (1337 and 1163), and read-only and immutable opens both succeeded with FTS queries returning hits. For the wiki, one real scaffold was copied to scratch (markdown only):rebuild-indexindexed 31 entities from 19 files, andwiki query— the FTS5 read path guarded here — returned correct results. Temporary stores deleted after measuring.Code-graph changes
This touches graph infrastructure (
db/sqlite.ts,db/database.ts,maintenance.ts) but adds no extractor, resolver, grammar or schema change, so the extractor-specific items below are marked N/A rather than ticked.mainLanguageExtractororFrameworkResolverinterface — N/A, no extractor or resolver added