Fix/FTS5 preflight coverage, store protection, and a discoverable telemetry opt-out

CHANGELOG.md

@@ -6,6 +6,13 @@ All notable changes to this project will be documented in this file.
 
 ### Fixed
 - `mex graph` now fails with an actionable message naming the running Node version when the built-in `node:sqlite` module lacks FTS5 support, instead of surfacing SQLite's raw `no such module: fts5` on the first schema statement that needs it. FTS5 availability is not guaranteed by every Node build/version inside the documented `engines` range (#110).
+- The FTS5 preflight now covers every consumer, not only `mex graph`'s writable open: read-only and immutable graph opens (`mex check`, `graph scope`/`query`/`get`, `impact`) and the wiki index, whose `wiki_fts` table has the same dependency. `mex wiki rebuild-index` reports the new `WIKI_INDEX_FTS5_UNAVAILABLE` diagnostic rather than `WIKI_INDEX_REBUILD_REQUIRED`, which would have sent users round a loop rebuilding an index no rebuild can fix (#110).
+- The wiki index's two direct read paths — contract status inspection and the read session — also preflight FTS5 now, instead of letting SQLite's raw error escape. Reachable by building the index on one Node and reading it on another (#110).
+- COMPATIBILITY.md documents the FTS5 requirement, a one-line command to check the Node you actually run, and that the v0.6.3 fallback predates the code graph. The preflight's error message pointed at a document that said nothing about FTS5 (#110).
+- `mex graph rebuild`/`refresh`/`repair` and `mex wiki rebuild-index` now ensure `.mex/.gitignore` exists before creating a store. Only `mex setup` did this, so building a store in a checkout that had never run setup left `graph.db`, `-wal` and `-shm` untracked, ready for the next `git add -A` to commit (#110).
+
+### Added
+- `mex telemetry disable` and `mex telemetry enable`, writing the same `~/.mex/config.json` key as `mex config set telemetry on|off`. `mex telemetry --help` and `mex telemetry status` now name the `DO_NOT_TRACK=1` and `MEX_TELEMETRY=0` env opt-outs and say which one is in effect; previously the only switch lived under `config` and the env vars appeared solely in the first-run notice (#110).
 
 ## [0.8.0] - 2026-09-02
 

COMPATIBILITY.md

@@ -2,7 +2,47 @@
 
 ## Runtime requirement
 
-mex 0.8.x requires Node.js 22.5 or newer. The code graph uses the built-in `node:sqlite` module; older Node releases are unsupported. Users who cannot upgrade Node can remain on mex v0.6.3, which supports Node.js 20 or newer.
+mex 0.8.x requires Node.js 22.5 or newer. The code graph and the wiki index use
+the built-in `node:sqlite` module; older Node releases are unsupported.
+
+Users who cannot upgrade Node can remain on mex v0.6.3, which supports Node.js
+20 or newer. Note what that costs: the code graph shipped in 0.7.0, so v0.6.3
+has no `mex graph`, no `mex impact`, and no code-node grounding. It is a
+scaffold-and-drift-checking release, not an older version of the same feature
+set.
+
+### SQLite FTS5
+
+**A supported Node version is necessary but not sufficient.** Both databases
+need SQLite's FTS5 full-text extension, and `node:sqlite` embeds whatever
+SQLite the Node binary was built with. FTS5 is a compile-time option that Node
+does not document or guarantee, so whether you have it depends on the *build*,
+not the version number alone — official builds, distro packages, and
+self-compiled Node can differ at the same version.
+
+Check the Node you actually run in one command:
+
+```console
+$ node --no-warnings -e "new (require('node:sqlite').DatabaseSync)(':memory:').exec('CREATE VIRTUAL TABLE t USING fts5(x)')" && echo "FTS5 ok"
+```
+
+Silence plus `FTS5 ok` means you are fine. `no such module: fts5` means that
+Node build cannot run the graph or the wiki index; install a different build or
+version of Node. mex preflights this itself, so `mex graph` and
+`mex wiki rebuild-index` name the problem and your Node version rather than
+failing with a bare SQLite error.
+
+Known data points, which are reports rather than a supported-range claim:
+
+| Node | Platform | FTS5 |
+|---|---|---|
+| 23.10.0 | Windows 11 | missing ([#110](https://github.com/mex-memory/mex/issues/110)) |
+| 24.11.0 | Windows 11 | present |
+
+`engines` stays at `>=22.5`: FTS5 does not track version order, so narrowing
+the range would lock out working builds without excluding broken ones. If you
+hit a build without it, please add it to the table via issue #110 — the sample
+is small, and that is the only thing that would justify a floor.
 
 This document defines `mex-agent`'s public contract: what's stable, what isn't,
 and what counts as a breaking change. It is intended for embedders — tools that

src/cli.ts

@@ -1040,7 +1040,74 @@ program
 // ── Telemetry ──
 const telemetryCmd = program
   .command("telemetry")
-  .description("Telemetry transparency commands");
+  .description("Telemetry transparency commands, including the opt-out")
+  .addHelpText(
+    "after",
+    "\nOpting out:\n"
+      + "  mex telemetry disable   Turn telemetry off for every project (~/.mex/config.json)\n"
+      + "  DO_NOT_TRACK=1          Standard cross-tool opt-out, honoured per invocation\n"
+      + "  MEX_TELEMETRY=0         mex-specific env opt-out, honoured per invocation\n"
+      + "\nEnvironment variables win over the stored setting. `mex telemetry status`\n"
+      + "reports which one is in effect.\n",
+  );
+
+/** Explain an opt-out reason in terms of the thing the user would have to change. */
+function describeTelemetryReason(reason: string | undefined): string {
+  switch (reason) {
+    case "DO_NOT_TRACK":
+      return "The DO_NOT_TRACK environment variable is set to 1.";
+    case "MEX_TELEMETRY":
+      return "The MEX_TELEMETRY environment variable is set to 0.";
+    case "dev":
+      return "This is a mex development checkout; telemetry never runs from one.";
+    case "config":
+      return "Stored in ~/.mex/config.json. Re-enable with `mex telemetry enable`.";
+    default:
+      return "";
+  }
+}
+
+/**
+ * `telemetry disable` / `enable` write the same `~/.mex/config.json` key as
+ * `mex config set telemetry off|on`.
+ *
+ * The duplication is the point. Issue #110 reported reaching for
+ * `mex telemetry disable`, getting `unknown command`, and then guessing at env
+ * var names — because the only switch lived under `config`, which is not where
+ * anyone looks for it. An alias costs nothing; a user who cannot find the
+ * opt-out costs trust.
+ */
+function setTelemetryEnabled(enabled: boolean): void {
+  try {
+    setGlobalConfigKey("telemetry", enabled ? "on" : "off");
+  } catch (err) {
+    console.error((err as Error).message);
+    process.exit(1);
+  }
+  console.log(`Telemetry ${enabled ? "enabled" : "disabled"} in ~/.mex/config.json`);
+
+  // Never claim an outcome the next invocation will contradict: an env opt-out
+  // outranks the stored value, and a dev checkout outranks both.
+  const active = isEnabled();
+  if (active.enabled !== enabled) {
+    const detail = describeTelemetryReason(active.reason);
+    console.log(
+      active.enabled
+        ? "Telemetry is still on for this project."
+        : `Telemetry stays off regardless of this setting. ${detail}`.trim(),
+    );
+  }
+}
+
+telemetryCmd
+  .command("disable")
+  .description("Turn telemetry off for every project (writes ~/.mex/config.json)")
+  .action(() => setTelemetryEnabled(false));
+
+telemetryCmd
+  .command("enable")
+  .description("Turn telemetry back on for every project")
+  .action(() => setTelemetryEnabled(true));
 
 telemetryCmd
   .command("inspect")
@@ -1073,9 +1140,12 @@ telemetryCmd
     const result = isEnabled();
     if (result.enabled) {
       console.log("Telemetry: enabled");
-    } else {
-      console.log(`Telemetry: disabled (reason: ${result.reason})`);
+      console.log("Turn it off with `mex telemetry disable`, DO_NOT_TRACK=1, or MEX_TELEMETRY=0.");
+      return;
     }
+    console.log(`Telemetry: disabled (reason: ${result.reason})`);
+    const detail = describeTelemetryReason(result.reason);
+    if (detail) console.log(detail);
   });
 
 // ── Config ──
@@ -1175,6 +1245,7 @@ program
     console.log("  mex watch              Install post-commit hook for auto drift score");
     console.log("  mex watch --interval   Run heartbeat every 30 minutes (or config value)");
     console.log("  mex watch --uninstall  Remove the post-commit hook");
+    console.log("  mex telemetry disable  Turn telemetry off (or DO_NOT_TRACK=1 / MEX_TELEMETRY=0)");
     console.log("  mex telemetry inspect  Show the exact telemetry payload (without sending)");
     console.log("  mex telemetry status   Show telemetry enabled/disabled and reason");
     console.log("  mex config set <k> <v> Set a global config value (e.g. telemetry off)");

src/graph/__tests__/database-fts5.test.ts

@@ -27,21 +27,12 @@ function fakeDb(execImpl: (sql: string) => void): SqliteDatabase {
 }
 
 /**
- * Load a fresh `database.js` whose `assertFts5Available` probes against
- * `fakeExec` instead of a real `:memory:` connection, by mocking the
- * `openSqlite` it imports from `sqlite.js`. `assertFts5Available` no longer
- * takes a `SqliteDatabase` parameter (PR #168 review: it must not touch the
- * caller's real graph database) — it opens its own throwaway connection
- * internally, so exercising the error paths now goes through this module
- * mock rather than an injected fake.
+ * The probe takes an injected opener precisely so its failure paths are
+ * reachable without an FTS5-less Node build — and without module mocking, which
+ * cannot reach a call `sqlite.ts` makes to its own `openSqlite`.
  */
-async function assertFts5AvailableWith(fakeExec: (sql: string) => void) {
-  vi.resetModules();
-  vi.doMock("../db/sqlite.js", () => ({
-    openSqlite: () => fakeDb(fakeExec),
-  }));
-  const fresh = await import("../db/database.js");
-  return fresh.assertFts5Available;
+function failingOpener(execImpl: (sql: string) => void) {
+  return (() => fakeDb(execImpl)) as unknown as Parameters<typeof assertFts5Available>[0];
 }
 
 describe("assertFts5Available", () => {
@@ -52,23 +43,38 @@ describe("assertFts5Available", () => {
     expect(() => assertFts5Available()).not.toThrow();
   });
 
-  it("raises an actionable, Node-version-specific message on the exact SQLite error from issue #110", async () => {
-    const probe = await assertFts5AvailableWith(() => {
+  it("raises an actionable, Node-version-specific message on the exact SQLite error from issue #110", () => {
+    const probe = failingOpener(() => {
       throw new Error("no such module: fts5");
     });
 
-    expect(() => probe()).toThrowError(
+    expect(() => assertFts5Available(probe)).toThrowError(
       new RegExp(`Node \\(${process.version.replace(/[.+]/g, "\\$&")}\\).*FTS5.*no such module: fts5`, "s"),
     );
   });
 
-  it("re-throws an unrelated exec failure unchanged, rather than misattributing it to FTS5", async () => {
-    const probe = await assertFts5AvailableWith(() => {
+  it("re-throws an unrelated exec failure unchanged, rather than misattributing it to FTS5", () => {
+    const probe = failingOpener(() => {
       throw new Error("database is locked");
     });
 
-    expect(() => probe()).toThrowError("database is locked");
-    expect(() => probe()).not.toThrow(/FTS5/);
+    expect(() => assertFts5Available(probe)).toThrowError("database is locked");
+    expect(() => assertFts5Available(probe)).not.toThrow(/FTS5/);
+  });
+
+  it("closes the throwaway probe connection on both the success and failure paths", () => {
+    let opened = 0;
+    let closed = 0;
+    const counting = (execImpl: (sql: string) => void) => (() => {
+      opened += 1;
+      return { ...fakeDb(execImpl), close: () => { closed += 1; } };
+    }) as unknown as Parameters<typeof assertFts5Available>[0];
+
+    assertFts5Available(counting(() => {}));
+    expect(() => assertFts5Available(counting(() => {
+      throw new Error("no such module: fts5");
+    }))).toThrow(/FTS5/);
+    expect(closed).toBe(opened);
   });
 });
 
@@ -91,45 +97,39 @@ describe("openGraphDatabase FTS5 preflight", () => {
     }
   });
 
-  it("closes the database handle when the FTS5 preflight fails, instead of leaking it open", async () => {
-    const root = mkdtempSync(join(tmpdir(), "mex-database-fts5-close-"));
+  it("never opens the store when the preflight fails, on the write path or either read path", async () => {
+    const root = mkdtempSync(join(tmpdir(), "mex-database-fts5-guarded-"));
     roots.push(root);
+    const dbPath = join(root, "graph.db");
+    openGraphDatabase(dbPath).close();
 
     vi.resetModules();
     vi.doMock("../db/sqlite.js", async () => {
       const actual = await vi.importActual<typeof import("../db/sqlite.js")>("../db/sqlite.js");
-      let realGraphDb: SqliteDatabase | undefined;
+      const storeOpens: string[] = [];
       return {
         ...actual,
+        // Only the probe's own :memory: connection is faked; a real store open
+        // is recorded so the test can prove it never happened.
+        assertFts5Available: () => actual.assertFts5Available((() => fakeDb(() => {
+          throw new Error("no such module: fts5");
+        })) as unknown as typeof actual.openSqlite),
         openSqlite: (path: string, options?: { readOnly?: boolean; immutable?: boolean }) => {
-          if (path === ":memory:") {
-            // The FTS5 preflight's own throwaway connection: fail it.
-            return {
-              prepare: () => {
-                throw new Error("not used by this test");
-              },
-              exec: () => {
-                throw new Error("no such module: fts5");
-              },
-              pragma: () => {},
-              transaction: <T>(fn: () => T) => fn(),
-              close: () => {},
-              open: true,
-            } satisfies SqliteDatabase;
-          }
-          realGraphDb = actual.openSqlite(path, options);
-          return realGraphDb;
+          storeOpens.push(path);
+          return actual.openSqlite(path, options);
         },
-        __getRealGraphDb: () => realGraphDb,
+        __storeOpens: () => storeOpens,
       };
     });
 
     const fresh = await import("../db/database.js");
-    const sqliteMock = (await import("../db/sqlite.js")) as unknown as {
-      __getRealGraphDb: () => SqliteDatabase | undefined;
-    };
+    const sqliteMock = (await import("../db/sqlite.js")) as unknown as { __storeOpens: () => string[] };
 
-    expect(() => fresh.openGraphDatabase(join(root, "graph.db"))).toThrow(/FTS5/);
-    expect(sqliteMock.__getRealGraphDb()?.open).toBe(false);
+    for (const options of [{}, { readOnly: true }, { readOnly: true, immutable: true }]) {
+      expect(() => fresh.openGraphDatabase(dbPath, options)).toThrow(/FTS5/);
+    }
+    // The preflight runs before the store is opened, so there is no handle to
+    // close — strictly better than opening one and closing it on the way out.
+    expect(sqliteMock.__storeOpens()).toEqual([]);
   });
 });

src/graph/db/database.ts

@@ -25,7 +25,12 @@ import {
   parseGraphSnapshot,
   serializeGraphSnapshot,
 } from "../snapshot.js";
-import { openSqlite, type SqliteDatabase } from "./sqlite.js";
+import { assertFts5Available, openSqlite, type SqliteDatabase } from "./sqlite.js";
+
+// The FTS5 preflight lives with the SQLite adapter (it describes the SQLite
+// build, not the graph) and is re-exported here for the callers and tests that
+// already reach for it through this module.
+export { assertFts5Available };
 
 /** The schema version this build writes/expects (matches schema.sql's seed). */
 export const DB_SCHEMA_VERSION = 4;
@@ -57,42 +62,6 @@ function configureReadOnlyConnection(db: SqliteDatabase): void {
   db.pragma("query_only = ON");
 }
 
-/**
- * Probe for FTS5 support and fail fast with an actionable message if it's missing.
- *
- * `node:sqlite`'s bundled SQLite is not guaranteed to be built with FTS5 on every
- * Node build/version, even within the range `package.json`'s `engines` documents
- * as supported (issue #110). Without this check, the first FTS5 statement in
- * `schema.sql` throws SQLite's raw `no such module: fts5`, which reads like a mex
- * bug rather than a Node/SQLite build limitation. Create-and-drop a throwaway
- * virtual table rather than querying `pragma_module_list`, since that pragma is
- * unavailable on some `node:sqlite` builds too and FTS5 usage is what actually
- * needs to work.
- *
- * FTS5 availability is a property of the SQLite build the running Node binary
- * embeds, not of any particular database file, so the probe runs against a
- * throwaway `:memory:` connection rather than the caller's real database.
- * Probing in place (an earlier version of this function took the caller's
- * `SqliteDatabase`) rewrote the on-disk graph on every successful open, which
- * broke a read-path non-mutation regression test in CI (PR #168 review).
- */
-export function assertFts5Available(): void {
-  const probe = openSqlite(":memory:");
-  try {
-    probe.exec("CREATE VIRTUAL TABLE __mex_fts5_probe USING fts5(x)");
-  } catch (error) {
-    const msg = error instanceof Error ? error.message : String(error);
-    if (!/fts5/i.test(msg)) throw error; // a different problem; surface it unchanged
-    throw new Error(
-      `Your Node (${process.version}) has SQLite built without FTS5 support, which mex's code graph ` +
-        "requires. Try a different Node build/version - see COMPATIBILITY.md for which versions are " +
-        `known to work. Underlying error: ${msg}`,
-    );
-  } finally {
-    probe.close();
-  }
-}
-
 /**
  * Open the graph DB at `dbPath`, creating the file + parent dir and applying the
  * schema when absent. Idempotent: re-opening an existing DB re-applies PRAGMAs
@@ -111,6 +80,13 @@ export function openGraphDatabase(
     mkdirSync(dir, { recursive: true });
   }
 
+  // Every graph open needs FTS5: writers apply `schema.sql`'s virtual tables,
+  // and readers query `nodes_fts` / `source_chunks_fts` (search, scope, impact,
+  // and the grounding checker). A store built on an FTS5-capable machine and
+  // copied to one without it fails on read, not on build, so the read-only and
+  // immutable paths need this preflight just as much as the writable one.
+  assertFts5Available();
+
   if (options.readOnly) {
     return options.immutable
       ? openImmutableGraphDatabase(dbPath)
@@ -120,7 +96,6 @@ export function openGraphDatabase(
   configureConnection(db);
 
   try {
-    assertFts5Available();
     initializeWritableGraphDatabase(db, readFileSync(schemaPath(), "utf-8"), options);
     return db;
   } catch (error) {