Unreadable files (macOS TCC/EPERM) are skipped silently: text search returns 0 results, reindex warnings carry no error, daemon status still reports ready

[mc-luisg]

Summary

On macOS, a daemon running without a Full Disk Access / Documents grant gets EPERM from open() for files under ~/Documents. read surfaces that error correctly, but every other surface degrades silently: text search returns count: 0 for strings that exist on disk, the incremental reindexer logs one warning per file with no error text, and daemon status keeps reporting ready. The net effect is an index silently frozen across a large part of the tree while agents keep trusting 0 results as "not present".

The permission problem itself is environmental and mine to fix. This issue is only about the silent degradation around it.

Versions / environment

• daemon v0.63.3+d480163, CLI v0.63.7+0988412 (tap now ships 0.63.8)
• macOS 26.6.2 (build 25G83), Darwin 25.6.0, arm64
• installed via Homebrew cask (zzet/tap)
• 1 tracked repo under ~/Documents/..., ~32k files, ~722k nodes

Caveat: the daemon that produced this evidence is 0.63.3, kept alive across an upgrade by the pre-postflight cask (the #609 leg that landed in 0.63.8). I have not verified the behaviour against 0.63.8, so some of this may already be improved — the reindex warning without error text is still present in the log format shown below.

What the daemon can and cannot read

Not a blanket denial, which is what makes it confusing. macOS records per-item TCC grants in the com.apple.macl xattr, so the daemon can still open files it previously wrote itself while everything else fails:

$ gortex call read --arg operation=file --json '{"target":{"file":"pkg-a/src/service.ts"}}'
{ "bytes": 68477, "content": "..." }        # item carries the daemon's MACL entry

$ gortex call read --arg operation=file --json '{"target":{"file":"pkg-b/src/Widget.tsx"}}'
Error: could not read file: open ~/Documents/<workspace>/pkg-b/src/Widget.tsx: operation not permitted

The same file reads fine from a shell as the same user (plain 0644, user-owned, no ACL, no file flags, not a symlink). EPERM rather than EACCES is the per-process TCC denial.

Symptom 1 — text search returns 0 instead of an error or a warning

A string that exists in 6 files on disk:

$ gortex call search --arg operation=text --arg query=some_identifier
{
  "count": 0,
  "matches": [],
  "query": "some_identifier",
  "scope_note": "0 results within the active scope (repo:<repo>) — widen with repo:\"*\" or pass an explicit repo:/project:"
}

Those files were never indexed, because they cannot be read. Nothing in the response distinguishes "not present" from "could not be read".

Worse, a symbol lookup for a symbol defined in one of those same unreadable files still returns a hit — served from a stale index built back when access worked. The graph is frozen at the last successful read and silently diverges from disk.

This is the part that actually causes harm: an agent treats count: 0 as ground truth.

Symptom 2 — reindex warnings carry no error text

~/.gortex/cache/daemon.log (43 MB) holds 6,676 of these across 5,796 distinct files, continuously over the last week:

{"level":"warn","ts":1787887896.023299,"caller":"indexer/incremental_batch.go:125","msg":"incremental reindex: file failed after retry","file":"~/Documents/<workspace>/pkg-b/.astro/content.d.ts"}

Every one of the 6,676 entries has an empty error field. The message says a file failed and was retried, but never why — so a week-old permission failure produced 43 MB of logs that could not be diagnosed from the logs themselves. Attaching the underlying error (operation not permitted) would have made this a 30-second diagnosis.

It is not a retry loop, for what it is worth: only 167 files were retried more than once (max 50x over two hours), tracking real edits.

Symptom 3 — no health surface reflects it

daemon status reports:

state     ready (warmup 3h21m)
search    sqlite-fts5  docs=328591  disk-resident

with the tracked repo showing its full file/node/edge counts. Nothing indicates that 5,796 files failed to index or that whole subtrees are stale. The MCP handshake line says the same: Index status: ready (phase "enrichment_complete").

Minimal repro

1. On macOS, ensure the gortex binary has no Full Disk Access grant (System Settings → Privacy & Security → Full Disk Access).
2. Start the daemon so it is its own responsible process — e.g. gortex daemon start, which daemonizes to PPID 1 and so does not inherit a terminal's TCC grant.
3. Track a repository under ~/Documents (a TCC-protected location).
4. Edit some files, or add new ones.
5. search for content in those files → count: 0, no error. daemon status → ready. daemon.log → file failed after retry with no reason.

Step 2 matters: a daemon auto-started from a terminal that does hold a grant behaves fine, which is why this can look intermittent across restarts.

What would help

1. Propagate the read error into the reindex warning — include the underlying error at indexer/incremental_batch.go:125. Cheapest fix, biggest diagnostic win.
2. A degraded-index signal queries can see. Track a per-repo count of files that failed to index and expose it: an unreadable: N / degraded row in daemon status and index_health, and ideally a field on query responses so a count: 0 from a partially unreadable repo is distinguishable from a genuine miss. scope_note would be a natural place for a one-liner.
3. Classify EPERM specifically. A permission denial is not transient and retrying does not help; it deserves a distinct, once-per-repo warning naming the likely cause on macOS ("N files unreadable — the daemon may need Full Disk Access") rather than N identical per-file warnings.

Related in spirit but different causes: #313 (partial SQL indexing, no warning), #311 (index_health over-reporting), #672 / #674 (search_text truncation indistinguishable from completeness).

Happy to test a patch — I have a machine reproducing this continuously.