While validating a rebuilt daemon over an existing store I hit a shape where an extractor version bump never lands: a repository with zero file churn takes the census_noop route on daemon warmup, and that route never consults the persisted extractor versions.
What I observed (Windows, current main at 116ff37 merged into my PR #677 branch; the mechanism is main's, my branch only raised the version number):
- stored snapshot in repo_index_state:
"csharp":15, no _post_extraction_policy key (written by the previous binary), so both staleness triggers were pending against the new binary (csharp at 16 plus policy epoch 2)
- daemon restart over the unchanged tree:
daemon: reconciled repo from snapshot route=census_noop changed=0 deleted=0 stale_files_reindexed=0
- the snapshot stayed at
"csharp":15 after warmup, and repeated restarts reproduce it - the graph keeps serving the old extraction until some source file happens to change
Where it forks (internal/indexer/multi.go:2995):
case churn == 0 && !idx.merkleEnabled():
route = "census_noop"
result, err = idx.cleanCensusResult(ctx, detected, start)
case churn == 0:
// The mtime census cannot prove a Merkle-enabled repository clean:
// a missing baseline or extractor-salt change still requires the
// content-addressed full-tree incremental check.
route = "incremental"The Merkle-enabled arm names extractor-salt drift as exactly the reason a clean census cannot be trusted, and routes to the full-tree incremental pass. The non-Merkle arm has no analogous guard: extractorVersionStaleLangSet is consulted only inside incrementalReindexPathsMode (indexer.go:5859), which census_noop never reaches. The R6 machinery from PR #432 and the restage tests in extractor_version_restage_test.go all pin the full-tree pass itself, which works; the reconcile routing just never runs that pass for a quiet repository unless Merkle is on.
Why I think this matters right now: the Julia extractor rollout is this exact shape. A store indexed by the previous release has .jl files with unchanged mtimes and a snapshot without a julia key. TestIncrementalReindex_NonMerkleNewlyTrackedLanguageRestages proves the full-tree pass restages it, but a default (non-Merkle) daemon upgrade with no pending edits takes census_noop and keeps the regex-era Julia graph across restarts indefinitely. Same for the EF Core csharp bump on any dormant C# repository, and for the _post_extraction_policy epoch itself.
Suggested fix, if you agree with the diagnosis: consult the staleness set during routing - when the stored snapshot reports version drift, treat churn==0 the same way the Merkle arm does and take the full-tree incremental route. The check is one stored-row read plus a map compare, and the restage re-stamps the snapshot so it fires once per bump. Happy to file a PR with a regression test at the reconcile-routing level if you want it.
I healed my own store with an untrack and retrack in the meantime, so nothing is blocked on my side.
While validating a rebuilt daemon over an existing store I hit a shape where an extractor version bump never lands: a repository with zero file churn takes the census_noop route on daemon warmup, and that route never consults the persisted extractor versions.
What I observed (Windows, current main at 116ff37 merged into my PR #677 branch; the mechanism is main's, my branch only raised the version number):
"csharp":15, no_post_extraction_policykey (written by the previous binary), so both staleness triggers were pending against the new binary (csharp at 16 plus policy epoch 2)"csharp":15after warmup, and repeated restarts reproduce it - the graph keeps serving the old extraction until some source file happens to changeWhere it forks (internal/indexer/multi.go:2995):
The Merkle-enabled arm names extractor-salt drift as exactly the reason a clean census cannot be trusted, and routes to the full-tree incremental pass. The non-Merkle arm has no analogous guard: extractorVersionStaleLangSet is consulted only inside incrementalReindexPathsMode (indexer.go:5859), which census_noop never reaches. The R6 machinery from PR #432 and the restage tests in extractor_version_restage_test.go all pin the full-tree pass itself, which works; the reconcile routing just never runs that pass for a quiet repository unless Merkle is on.
Why I think this matters right now: the Julia extractor rollout is this exact shape. A store indexed by the previous release has .jl files with unchanged mtimes and a snapshot without a julia key. TestIncrementalReindex_NonMerkleNewlyTrackedLanguageRestages proves the full-tree pass restages it, but a default (non-Merkle) daemon upgrade with no pending edits takes census_noop and keeps the regex-era Julia graph across restarts indefinitely. Same for the EF Core csharp bump on any dormant C# repository, and for the
_post_extraction_policyepoch itself.Suggested fix, if you agree with the diagnosis: consult the staleness set during routing - when the stored snapshot reports version drift, treat churn==0 the same way the Merkle arm does and take the full-tree incremental route. The check is one stored-row read plus a map compare, and the restage re-stamps the snapshot so it fires once per bump. Happy to file a PR with a regression test at the reconcile-routing level if you want it.
I healed my own store with an untrack and retrack in the meantime, so nothing is blocked on my side.