Skip to content

resolver: derived tests edges never re-resolve, and test-linkage skips unresolved calls - #679

Open
pbednarcik wants to merge 5 commits into
zzet:mainfrom
pbednarcik:fix/test-edges-unresolved-resolution
Open

resolver: derived tests edges never re-resolve, and test-linkage skips unresolved calls#679
pbednarcik wants to merge 5 commits into
zzet:mainfrom
pbednarcik:fix/test-edges-unresolved-resolution

Conversation

@pbednarcik

Copy link
Copy Markdown
Contributor

While validating other work I hit a counted-fixture tripwire: an
extension-stamped edge appeared where the binder provably refuses to
bind. The artifact was odd enough to chase down: kind tests, origin
ast_inferred, confidence 0.75, Meta resolution: extension_method,
pointing from a plain method to the one extension method sharing its
name, while the calls edge at the same call site stayed honestly
unresolved.

Root cause, in three steps:

  1. The test-linkage pass clones a test caller's calls edges into
    EdgeTests, and it clones unresolved calls too. The clone carries none of
    the original's Meta, so the receiver evidence (receiver_shape and
    friends) is stripped.

  2. resolveEdge has no route for EdgeTests, so an unresolved clone
    falls into the member-call case like any other *. target.

  3. On the real calls edge the extension shape guard refuses (the
    receiver is a List<int>, the candidate is this List<string>). The
    naked clone has no receiver evidence at all, so nothing can refuse, and
    the untyped pool-unique fallback binds it at 0.75.

The result is a tests edge asserting the exact bind the guarded calls
edge exists to prevent. The exposure is not small: my stores carry ~29k
tests edges pointing at unresolved targets (assertion-framework calls
from test files dominate), each one an evidence-stripped clone a future
incremental re-resolve can convert the same way. Cold builds emit test
edges after resolution settles, which is why the shape only surfaces on
incremental paths.

Two commits, one per layer:

  1. resolveEdge gains an EdgeTests gate, the same shape as the existing
    type-position gate (extends/implements/returns/typed_as): a derived edge
    never routes through call resolution. The tests layer follows its calls
    edge; the resolver never binds it independently. This half also protects
    the clones already sitting in existing stores.

  2. The test-linkage emitters skip unresolved calls entirely. A tests
    edge to an unresolved stub can never say what is tested, so the clones
    were dead weight at scale even before the wrong-bind hazard; a call that
    resolves later links on the next reconciliation of its caller.

Both fixes are RED-first: the resolver test reproduces the field
artifact end to end (a C# fixture through the real extractor, the tests
clone added at the same site, ResolveAll binds it pre-fix while the
calls-edge control stays refused), and the indexer test pins that only
resolved subjects clone.

Verification: the full race suite's fail-name set is identical to my
Windows baseline at the merge base (one known-flaky baseline name
happened to pass). The branch is based on 3eb51c6 and merges cleanly
onto current main.

The test-linkage pass clones a test caller's calls edges into the tests
layer meta-free, so an unresolved clone reaching the resolve cascades is
re-bound WITHOUT the original's receiver evidence - every receiver-gated
guard is bypassed. Observed in the field: a List<int> call site whose
calls edge the extension shape guard correctly refuses had its naked
tests clone bound to the repo's lone `this List<string>` extension by
the untyped pool-unique fallback at 0.75. Same gate shape as the
type-position kinds: the tests layer follows its calls edge, the
resolver never binds it independently.
A tests edge to an unresolved stub can never say what is tested, is dead
weight at scale (tens of thousands on a large store - assertion-framework
calls from test files dominate), and is the naked-clone hazard the
resolver-side gate closes from the other end. Only resolved subjects
clone; a call that resolves later simply links on the next
reconciliation of its caller.

@zzet zzet left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this — the underlying receiver-evidence bug is real, and preventing unresolved calls from being projected is the right direction. I found several production paths that the current tests do not cover, though, so I think these need to be addressed before merge.

1. P1: enforce the EdgeTests guard before every resolver path

The new guard in resolver.go runs after inline LSP resolution. Deferred LSP also collects EdgeTests before reaching this switch, and CrossRepoResolver has an independent resolveEdge with no equivalent guard.

Exact-head reproductions:

ResolveFile + fake LSP: EdgeTests bound to target.ts::Foo
ResolveAll + fake LSP:  EdgeTests bound to target.ts::Foo
CrossRepo ResolveAll:   EdgeTests bound to repoB/target.ts::Foo

This contradicts the PR's central invariant and means persisted unresolved test edges can still be independently misbound.

Suggested fix: use a shared resolver-eligibility predicate before inline/deferred LSP and in CrossRepoResolver. Please add ResolveFile + LSP, ResolveAll + LSP, and cross-repository regression tests.

Relevant locations:

  • internal/resolver/resolver.go:3093-3098
  • internal/resolver/lsp_resolve.go:277
  • internal/resolver/cross_repo.go:1044

2. P1: reconcile calls that become resolved later

Skipping an unresolved call in internal/indexer/test_edges.go:716 can permanently omit a valid test projection.

Reproduction:

  1. Seed TestFoo -> unresolved::Foo as an EdgeCalls and run test projection; no EdgeTests is emitted.
  2. Add pkg/foo.go::Foo.
  3. Run ResolveIncomingForFile("pkg/foo.go"); the call now resolves to Foo.
  4. Run a declaration-only derived plan for pkg/foo.go.
  5. The expected EdgeTests(TestFoo, Foo) is still absent.

The exact-head regression fails with:

resolved test call has no EdgeTests projection

A declaration-only plan does not reconcile test edges, and the caller file is not necessarily revisited later. Suggested fix: have resolution report the source files of retargeted EdgeCalls, merge that frontier into DerivedInvalidatesTests, and reconcile those callers after resolution. Please cover add, rename/delete/rebind, and cross-repository cases.

3. P2: clean up legacy unresolved EdgeTests on upgrade

The new filter prevents new unresolved clones but does not remove rows persisted by older versions. Whole-graph emission only adds current projections, while warm/no-change startup may skip file-scoped reconciliation entirely. An exact-head test seeded with a legacy unresolved EdgeTests row retains it after reconciliation.

For 29,000 legacy rows, the measured ResolveAll overhead was approximately:

Store Time Allocated
In-memory graph 102.7 ms 69.3 MB
SQLite 915.5 ms 50.3 MB

Suggested fix: add a versioned migration that deletes unresolved EdgeTests, or evict and rebuild the applicable projection set. Please include an SQLite seed/close/reopen upgrade test.

The two tests added by this PR pass, as do the hosted checks and go vet, but they cover only immediate emission and the no-LSP heuristic resolver path. I found no security, hardcoded-secret, off-by-one, or dead-code issue in the change.

The EdgeTests refusal sat inside resolveEdge's heuristic dispatch, so
three paths still bound derived tests clones by name with no receiver
evidence: the inline LSP hot-path runs before the dispatch, bulk-mode
ResolveAll collects its deferred LSP batch before resolveEdge sees the
clone at all, and CrossRepoResolver has an independent resolveEdge with
no equivalent case.

resolutionExempt is now the shared eligibility predicate: resolveEdge
consults it first (ahead of the inline LSP lookup), lspDeferTarget
refuses exempt edges at collection, and the cross-repository resolveEdge
refuses them before its tiers. Regressions pin all three paths:
ResolveFile with an answering LSP helper, ResolveAll through the
deferred batch, and the cross-repository same-repo name tier.
The test projection skips unresolved calls, so a call that binds AFTER
its caller was projected needs the caller reconciled - but the
definition-side derived plan never names the caller file, and its flags
can be declaration-only, so the coordinator skipped test projection
entirely. The valid EdgeTests was then permanently absent: the caller
file is not necessarily ever revisited.

Resolution passes now report their retargeted test-call frontier: every
core apply site (the parallel ResolveAll batch, the incremental/file
apply, the deferred LSP batch, and the cross-repository resolveEdge)
notes the caller file of a calls edge it bound when that caller is
test-classified (test file path, or an is_test-stamped source symbol -
annotation-marked tests live outside test-named paths). The frontier is
instance-scoped and drained destructively, so throwaway cold-warmup
resolvers leak nothing and the whole-graph test emission that follows
warmup stays the single owner of that scale.

Consumers: the derived coordinator drains the per-repo resolvers and
reconciles the frontier regardless of plan flags; the receipt-exact
catch-up lanes (master file/incoming resolves, both cross-repository
wrappers) drain their own resolver instances and reconcile inline,
since no global test pass runs behind them.

Regressions: the declaration-only plan shape (the reported repro), the
incremental add and rebind flows end to end (rebind also pins that the
stale projection to the evicted subject disappears), and the
cross-repository inbound test call.
The emission guard prevents new unresolved EdgeTests clones, but rows
persisted by older versions stay: whole-graph emission only adds
current projections, warm startup may skip file-scoped reconciliation
entirely, and the resolver now refuses to bind a tests edge - so a
pre-guard store keeps its naked stub rows and their resolver-scan cost
forever.

Schema v13 deletes tests-kind edges whose target is an unresolved stub
in either spelling (bare and repo-prefixed COPY-rewrite), leaving
pending calls and resolved projections untouched. Seed/close/reopen
upgrade regression included.
@pbednarcik

Copy link
Copy Markdown
Contributor Author

All three findings are addressed at exact head, each RED-first with your
reproductions as fixtures.

  1. The refusal now covers every resolution path through one shared
    predicate (resolutionExempt): resolveEdge consults it first, ahead of
    the inline LSP hot-path; lspDeferTarget refuses exempt edges at
    collection time, since the bulk batch is gathered before resolveEdge
    sees the clone; and the cross-repository resolveEdge refuses them ahead
    of its tiers. Your three reproductions are the regression tests -
    ResolveFile with an answering LSP helper, ResolveAll through the
    deferred batch, and the cross-repository same-repo name tier - and the
    old dispatch-internal case is deleted in favor of the shared predicate.

  2. Later-resolved calls now reconcile. Resolution passes report their
    retargeted test-call frontier: the core apply sites (the parallel
    ResolveAll batch, the incremental and file-scoped applies, the deferred
    LSP batch, and the cross-repository pass) note the caller file of every
    calls edge they bind whose caller is test-classified - by test file
    path, or by an is_test stamp on the source symbol, so annotation-marked
    tests outside test-named paths are covered. The frontier is
    instance-scoped and drained destructively: the derived coordinator
    drains the per-repo resolvers and reconciles those callers regardless
    of plan flags (your declaration-only shape is the regression, failing
    with your exact message before the fix), and the catch-up lanes with no
    global test pass behind them - the master file resolve and both
    cross-repository wrappers - drain their own instances and reconcile
    inline. Cold warmup stays unchanged: its throwaway resolver instances
    leak nothing, and the whole-graph emission that follows warmup remains
    the single owner of that scale. End-to-end regressions cover the add
    and rebind flows on real files (rebind also pins that the stale
    projection to the evicted subject disappears) and the cross-repository
    inbound test call.

  3. Legacy rows purge on upgrade. Schema v13 deletes tests-kind edges
    whose target is an unresolved stub, in both the bare and the
    repo-prefixed COPY-rewrite spellings, leaving pending calls and
    resolved projections untouched. The seed/close/reopen upgrade test you
    asked for is included; the in-memory backend needs no counterpart since
    it always loads from a migrated store or a fresh index.

Full race suite on Windows: the fail-name set matches my baseline except
one known timing flake in each direction (a Swift tight-window test that
reproduces standalone on a clean checkout of the base, and one baseline
flake that happened to pass this run). The resolver, indexer, and
store_sqlite packages are clean under -race.

@zzet zzet left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at 5910a1de. P1-A is properly fixed, and the store data backs the premise more strongly than the PR claims. Two things still need work: one production lane where P1-B still reproduces, and the test integrity of the reconcile commit.

Measured read-only against a 20-repo store (9.5 GB, 7,479,444 edges):

count
tests edges 104,972
→ unresolved target (purged by v13) 42,962 (41% of the whole tests layer)
migration predicate cost 1.35 s

EXPLAIN QUERY PLAN gives SEARCH edges USING INDEX edges_by_kind, so the leading-wildcard LIKE never scans the 7.5M-row table. The startup-cost concern from the last round is settled — and 42,962 is well above the ~29k the description estimates.


1. BLOCKING — P1-B still reproduces on the whole-graph master lane

runMasterResolveFiles drains the frontier (multi.go:939). runMasterResolveHookedContext does not: it runs master.ResolveAllContext at multi.go:910 and returns at :923 with no drain. Every whole-graph master lane funnels through it — runMasterResolve (:875), runMasterResolveHooked (:881), and :988. newMasterResolver returns a fresh resolver per call, so that frontier is garbage-collected unread.

This is reachable on a routine path: resolveDeferredMutations' fullFallback (multi.go:783, calling through at :793) is the fail-closed whole-graph catch-up taken whenever a mutation receipt comes back incomplete. runCrossRepoResolveContext runs right after and does drain — but only its own CrossRepoResolver frontier; it never sees what the master bound.

Reproduce

Drop this in internal/indexer/ and run GOWORK=off go test -run TestReproMasterResolveAllDropsRetargetFrontier ./internal/indexer/ -v:

func TestReproMasterResolveAllDropsRetargetFrontier(t *testing.T) {
	build := func() (*MultiIndexer, *graph.Edge, graph.Store) {
		g := graph.New()
		g.AddNode(&graph.Node{ID: "repoA/pkg/foo_test.go", Kind: graph.KindFile, Name: "foo_test.go", FilePath: "repoA/pkg/foo_test.go", Language: "go", RepoPrefix: "repoA", WorkspaceID: "ws"})
		g.AddNode(&graph.Node{ID: "repoA/pkg/foo_test.go::TestFoo", Kind: graph.KindFunction, Name: "TestFoo", FilePath: "repoA/pkg/foo_test.go", Language: "go", RepoPrefix: "repoA", WorkspaceID: "ws"})
		g.AddNode(&graph.Node{ID: "repoA/pkg/foo.go", Kind: graph.KindFile, Name: "foo.go", FilePath: "repoA/pkg/foo.go", Language: "go", RepoPrefix: "repoA", WorkspaceID: "ws"})
		g.AddNode(&graph.Node{ID: "repoA/pkg/foo.go::Foo", Kind: graph.KindFunction, Name: "Foo", FilePath: "repoA/pkg/foo.go", Language: "go", RepoPrefix: "repoA", WorkspaceID: "ws"})
		call := &graph.Edge{From: "repoA/pkg/foo_test.go::TestFoo", To: graph.UnresolvedMarker + "Foo", Kind: graph.EdgeCalls, FilePath: "repoA/pkg/foo_test.go", Line: 5}
		g.AddEdge(call)
		if _, emitted := markTestSymbolsAndEmitEdges(g); emitted != 0 {
			t.Fatalf("fixture: unresolved call must not project, emitted %d", emitted)
		}
		return NewMultiIndexer(g, newTestRegistry(), search.NewNull(), nil, zap.NewNop()), call, g
	}

	t.Run("control_runMasterResolveFiles", func(t *testing.T) {
		mi, call, g := build()
		mi.runMasterResolveFiles([]string{"repoA/pkg/foo.go"}, false)
		if call.To != "repoA/pkg/foo.go::Foo" {
			t.Fatalf("fixture: call must bind, got %q", call.To)
		}
		if e := findEdge(g, graph.EdgeTests, "repoA/pkg/foo_test.go::TestFoo", "repoA/pkg/foo.go::Foo"); e == nil {
			t.Fatalf("control: expected EdgeTests projection")
		}
	})

	t.Run("subject_runMasterResolve_wholegraph", func(t *testing.T) {
		mi, call, g := build()
		mi.runMasterResolve(nil, false) // the fullFallback lane (multi.go:793)
		if call.To != "repoA/pkg/foo.go::Foo" {
			t.Fatalf("fixture: call must bind, got %q", call.To)
		}
		if e := findEdge(g, graph.EdgeTests, "repoA/pkg/foo_test.go::TestFoo", "repoA/pkg/foo.go::Foo"); e == nil {
			t.Fatalf("BUG: master ResolveAll bound the test call but no EdgeTests was reconciled")
		}
	})
}

At 5910a1de:

--- PASS: TestReproMasterResolveAllDropsRetargetFrontier/control_runMasterResolveFiles
--- FAIL: TestReproMasterResolveAllDropsRetargetFrontier/subject_runMasterResolve_wholegraph
    zz_repro_test.go:48: BUG: master ResolveAll bound the test call but no EdgeTests was reconciled

Fix

One line in runMasterResolveHookedContext, mirroring :939:

 		zap.Int("pending_admitted", stats.PendingAfter),
 		zap.Error(err))
+	mi.reconcileRetargetedTestCalls(master.TakeRetargetedTestCallFiles())
 	return err
 }

Verified: the repro goes green, and the full internal/indexer suite stays green with it applied (ok ... 150.625s). Worth landing the repro above as the regression test so the lane stays pinned.


2. BLOCKING — two of the four new reconcile tests pass with the entire PR reverted

Neuter all three fixes (resolutionExemptreturn false; both TakeRetargetedTestCallFilesreturn nil) and run the four reconcile tests:

--- PASS: TestIncrementalReindex_LaterResolvedCallGainsTestsProjection   <- vacuous
--- PASS: TestIncrementalReindex_RebindMovesTestsProjection             <- vacuous
--- FAIL: TestDeclarationOnlyPlanReconcilesLaterResolvedTestCall        <- genuinely pins the fix
--- FAIL: TestCrossRepoResolveReconcilesRetargetedTestCall              <- genuinely pins the fix

Both passers drive the change through IncrementalReindexPaths, whose derived plan names the caller file — so the pre-existing evict-and-re-emit sweep in markTestSymbolsAndEmitEdgesForFilesLocked (test_edges.go:335) produces the expected result whether or not the retarget frontier exists. This includes the stale-edge assertion in the rebind test, and it includes the headline P1-B test.

The two that fail are good; keep them. For the other two: either drop them as redundant, or re-point them at a plan shape that does not name the caller file (the declaration-only shape the third test already uses), or assert directly that the drained frontier contains foo_test.go.


3. BLOCKING (small) — the warm-path emission guard has no coverage

test_edges.go:319, the unresolved-call skip inside markTestSymbolsAndEmitEdgesForFilesLocked, is the guard on the incremental path taken on every warm save. Replace its condition with if false and the whole suite still passes:

ok  github.com/zzet/gortex/internal/indexer  107.712s

The only emission test, TestMarkTestSymbolsAndEmitEdges_SkipsUnresolvedCallTargets, calls markTestSymbolsAndEmitEdges(g) with no files, which routes to emitTestEdgesAndPersistLocked and pins only the :712 guard.

Fix: a sibling test calling markTestSymbolsAndEmitEdgesScoped(g, nil, "pkg/foo_test.go") on the same two-call fixture, asserting emitted == 1 and no unresolved EdgeTests.


Non-blocking

a. Already-bound legacy clones survive the purge

The DELETE matches only an unresolved to_id, so a clone the old resolver already bound — the exact motivating artifact — survives forever. Real, but small: 22 same-repo tests rows with a resolved target and no sibling calls edge in that 20-repo store. They do include genuine cross-language impossibilities, e.g.

torture.rb::Documented.wire             -> react-native-bridge.test.ts::method
CustomerMgt.Codeunit.al::CreateCustomer -> SharedPointer.h::TSharedPtr.Get

Caution — do not widen the predicate with a NOT EXISTS sibling-calls probe. I tried exactly that; it deletes 9,724 legitimate cross-repo projections (9,724 of the 9,746 orphan rows are cross-repo) and it fails your own TestOpenPurgesLegacyUnresolvedTestsEdges, whose healthy fixture row has no sibling calls edge either. If you want the residue, gate on same-repo and cross-language; otherwise accepting it is defensible at this scale.

b. Cold index parks the whole test corpus on the frontier

indexer.go:3768 (idx.resolver.ResolveAll()) notes every bound test caller, and no cold path drains it. The first later incremental save drains all of them into markTestSymbolsAndEmitEdgesScoped under ResolveMutex — on this repo that is ~2.4k test files re-projected inside one save. Cheap fix: discard the frontier at the end of the cold whole-graph path, which already runs its own graph-wide projection.


Verified good

  • resolutionExempt being blanket on Kind == EdgeTests is correct, not over-broad: internal/indexer/test_edges.go is the only producer of that kind in the tree; every other reference is a consumer.
  • The two LIKE patterns are exactly equivalent to graph.IsUnresolvedTarget (HasPrefix("unresolved::") || Contains("::unresolved::")), with no LIKE metacharacters.
  • Merges cleanly onto current main (217b60ca); 656 -race tests pass across resolver / indexer / store_sqlite on the merged state.
  • Two constructs that look wrong are right: mi.mu.RLock() in drainRetargetedTestCallFiles is not reentrant-unsafe (the same function already RLocks earlier), and the len(files) > 0 && guard is correct — zero files means a whole-graph pass, so merging the retargeted list in would downgrade it to scoped.

Heads-up unrelated to correctness: #681 also bumps currentSchemaVersion 12 -> 13. Not silent — validateSchemaMigrations enforces strictly-ascending versions and highest == current, so whoever lands second just renumbers to 14.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants