search_text: limit is clamped to 1000 and the clamp also truncates count, so a capped result is indistinguishable from a complete one

[chenroth-island]

Describe the bug

search_text clamps the requested limit to 1000 (internal/mcp/tools_search_text.go:52). The clamp bounds the matches array and the reported count, and sets no truncation flag — so a caller cannot tell a capped result from a complete one. There is no cursor in the response to page past it, and path filtering is applied after truncation, so it cannot recover the remainder either.

This matters more for search_text than for the two siblings sharing the same clamp (tools_find_declaration.go:71, tools_graph_query.go:56): it is the literal-search backbone agents use in place of grep, where an unbounded sweep is the normal request. The byte budget (max_bytes) already bounds response size, and does so detectably via _truncated_by_budget.

(Repo and workspace names below are anonymized.)

To Reproduce

1. Track a large TypeScript monorepo (63,820 files here, called repo-ts below).
2. Run:

   gortex call search --arg operation=text --arg query=Typography \
     --arg 'limit:=100000' --arg 'max_bytes:=20000000' \
     --index ~/work/repo-ts --format json
   

3. Response: count: 1000, 1000 matches, no _truncated_by_budget, no cursor.
4. Ground truth: git grep -nP Typography | grep -cE '^[^:]*:[0-9]+:' → 8963.

Expected behavior

Either the requested limit is honored, or the response says it was bound — a flag, or a count that stays exact the way the byte budget keeps it exact. Silent loss of 7,963 of 8,963 matches, with a count that corroborates the wrong number, is the one failure mode a grep replacement cannot have.

Environment:

• OS: macOS 26.5.1 (25F80)
• Go version: 1.26.5
• Gortex version: v0.63.7+0988412

Additional context

Measured losses, all with the flag unset:

query	returned	real	lost
repo-ts / Typography	1000	8963	7963
repo-ts / useState	1000	5071	4071
repo-go / (4-char term)	952	3415	2463
repo-ts / useEffect	1000	3156	2156
repo-agent / Installer	1000	2271	1271
repo-infra / namespace	998	2088	1090

Note 952 and 998 — the cap is approached rather than always hit exactly, so even len(matches) == 1000 is not a reliable client-side detector.

Three workarounds checked and ruled out:

• limit:=100000 — clamped before the search runs.
• Pagination — no cursor emitted; response keys are only count, matches, query.
• path slicing — filterTextMatchesByPath runs after GrepTextForRepos has already truncated, so a slice returns whatever survived the global 1000, not the subtree's real total.

Git archaeology, in case it is simply vestigial: the clamp arrived in 16b624dd (2026-05-20), the commit that first added search_text, and is unchanged across the 3,329 commits since. budget-by-default with graceful degradation, pagination, sparse fieldsets (b36c307b) landed 2026-05-09 — eleven days earlier — so the byte budget already existed when the clamp was written.

Happy to send a PR. The shape I would propose is a GORTEX_SEARCH_TEXT_MAX_LIMIT override defaulting to 1000, so existing behavior is unchanged and an explicit unbounded sweep becomes expressible.

---

Update: validated with a local patch, including the "what if it is unbounded" case

I built the proposed change (env override, default still 1000) and measured it, specifically to answer the reasonable objection that lifting the ceiling invites huge responses. Short version: the ceiling was pure loss — removing it costs nothing, and max_bytes takes over as the binding constraint while remaining detectable.

Correctness: the clamp was the only thing missing those matches

Same three queries, GORTEX_SEARCH_TEXT_MAX_LIMIT=200000:

query	before	after	git grep	false positives
Typography	1000	8681	8963	0
useState	1000	5058	5071	0
useEffect	1000	3134	3156	0

Every remaining difference is content the graph deliberately does not index, verified by diffing the hit sets: .claude/ and .cursor/ markdown (265 + 14 of the 282), one vendored *.generated.ts, and Nx __tmpl__ template files. extra = 0 on all three — the result is a strict subset of grep, never a superset, so nothing is invented.

Performance: honoring the request is faster than grep, at every size tested

Warm, median of 3, whole-repo scope:

query	matches	gortex	git grep
Typography	8681	0.72s	1.63s	2.3x faster
useState	5058	0.75s	1.71s	2.3x faster
useEffect	3134	0.69s	1.64s	2.4x faster

So the cap was not buying latency. It discarded 7,681 matches on Typography to save nothing measurable.

The pathological case: export, 109,604 real hits

This is the query I would worry about as a maintainer, so I ran it with the limit at 200,000 and max_bytes at the default-ish 20 MB:

count = 108747
returned = 108143
_truncated_by_budget = True
response = 21.1 MB
elapsed = 8.7s
daemon RSS = 1.3 GiB -> 1.9 GiB, returned afterward

Three things this shows:

1. Unbounded limit does not mean an unbounded response. max_bytes bound it, exactly as designed.
2. And it bound it detectably — _truncated_by_budget: True was set, which is precisely what the 1000 clamp fails to do. The two ceilings are not redundant with each other: one is honest and one is not.
3. Memory stayed bounded — roughly 600 MiB of transient allocation for a 108k-match response, released afterward. No OOM, no daemon instability.

In other words, removing the clamp does not remove the guardrail; it removes the silent guardrail and leaves the honest one in charge. A caller who asks for 200,000 matches and hits the byte budget is told so; today a caller who asks for 200,000 and gets 1000 is not.

Suggested shape (unchanged, now measured)

if max := searchTextMaxLimit(); limit > max {
    limit = max
}

with searchTextMaxLimit() reading GORTEX_SEARCH_TEXT_MAX_LIMIT and defaulting to searchTextMaxLimitDefault = 1000. Zero behavior change unless opted into. Happy to open the PR against main with the comment explaining why the byte budget is the appropriate ceiling and this one is not.

If you would rather keep a hard ceiling, raising the default to something like 100,000 and setting a truncation flag when it binds would also fix the correctness problem — the flag matters more than the number.

Environment for the above: macOS 26.5.1, Go 1.26.5, patch applied to main at 27b83069; five-repo workspace, 2,158,904 nodes / 10,388,123 edges; the TypeScript repo measured is 63,820 files.

[chenroth-island]

Correction to my own numbers above, since it affects one of the arguments I made.

I benchmarked against git grep -nP. The client I actually run this through uses ripgrep, which is a much harder baseline — git grep sits at a flat ~1.4-1.7s on this repo regardless of query, so ratios against it overstate the win by roughly 4-6x.

Re-measured warm against ripgrep 15.1.0, median of 3, same three queries:

query	matches	gortex	ripgrep
Typography	8681	0.70s	1.43s
useState	5058	0.81s	1.43s
useEffect	3134	~0.7s	1.49s

So "2.3x faster than grep" should read ~1.8-2.0x faster than ripgrep for these large result sets. Still faster, and the substantive point is unchanged: honoring the requested limit is not slower than the clamp, so the ceiling was not buying latency.

Two things worth adding, both of which cut against a naive "gortex is just faster" reading:

The cost model is two-variable. Gortex is a fixed ~0.1s round trip plus serialization proportional to the number of hits; ripgrep is proportional to corpus size and nearly flat in hits. Across a five-repo workspace the median came out at 1.8x with a range of 0.3x-7.3x, and the crossover is around 20-30k files. On the 63,820-file repo gortex is 6-7x faster on a focused query; on a 5,449-file repo it is 0.3x on a 2,242-hit one. That makes the unclamped path more valuable rather than less, because the queries that benefit most from a raised ceiling are exactly the large-result ones where serialization dominates — and 0.70s for 8,681 matches is the number to weigh, not the ratio.

Two ripgrep measurement traps, in case they save someone else a wrong conclusion: rg with no path operand reads stdin, so from a subprocess with redirected stdin it searches nothing, exits in 0.01s and reports zero matches (looks like rg is 100x faster); and --no-ignore-vcs sends it into node_modules/, giving 14,004 hits and 10s where the real answer is 8,406.

The pathological-case data (export, 108,143 returned, _truncated_by_budget: True, 21.1 MB, daemon 1.3 -> 1.9 GiB and released) is unaffected — that was never a comparative measurement.

[chenroth-island]

Split the disclosure half out into #674, since it stands alone: raising this ceiling without adding a truncation flag just relocates the silent cliff, so #674 applies whatever number you land on here. Patch is in that issue and verified; the two changes are independent.