Add exact-match mode to /lookup and /bulk-lookup, and run bulk lookups in parallel

[gaurav]

Adds an exact parameter to /lookup and /bulk-lookup, so that callers who already know the string they are looking for can ask for whole-string matching instead of the default tokenized search, and sends the individual lookups behind /bulk-lookup concurrently instead of one at a time. The motivating case is named entity recognition pipelines, which resolve large numbers of exact strings in bulk and pay for relevance ranking they never use.

Closes #258.

What's here

Exact matching. When exact is set, the eDisMax query is bypassed entirely in favour of a Solr filter query against the *_exactish fields. Those are indexed with a keyword tokenizer and a lowercase filter, so the whole string must match, case-insensitively — parkinson does not match the label parkinsonian disorder, but Parkinsonian Disorder does.

exact	Matches against
label	The preferred name only (preferred_name_exactish)
synonyms	The synonyms only (names_exactish)
any	Either
(omitted)	Nothing changes — the usual tokenized eDisMax search

label and synonyms can genuinely disagree, because a concept's preferred name is not necessarily one of its synonyms — HP:0001300 is preferred_name="parkinsonian disorder" with names=["Parkinsonian disease"], and neither string appears in the other field. That is why the two modes are separate rather than one boolean, and it is also a data problem in its own right: NCATSTranslator/Babel#1073 proposes putting each clique's preferred name into its synonyms.

Interaction with the other parameters. Three of them behaved badly with exact, all silently, and all now do something defensible:

• autocomplete=true was ignored. It treats the final word as an incomplete prefix, which contradicts matching the whole string, so the combination is now a 400 rather than a request that quietly does something else. The rejection lives in lookup(), so it applies to /bulk-lookup too.
• The smart-quote rewrite (Queries containing ’ don't work correctly #176) was applied to the exact query. The *_exactish fields do no punctuation folding, so the indexed value keeps whichever quote characters Babel emitted; folding the query searched for a string the caller had not typed, and put any label containing a typographic quote out of reach. Exact mode no longer folds. The default path is unaffected, because StandardTokenizer discards the punctuation anyway.
• highlighting=true returned empty arrays, because the query is *:* and the exactish fields are not stored, so Solr had nothing to mark up. Since every exact match is a whole-value match, the highlighting is now synthesized from the returned document — the entire matching name, in its own capitalisation, wrapped in the same <strong> tags the default path asks Solr for. Callers no longer have to care which mode produced the field.

Parallel bulk lookup. bulk_lookup() issues its Solr queries with asyncio.gather() rather than awaiting them one at a time, bounded by a semaphore of SOLR_MAX_CONCURRENT_LOOKUPS (default 100). The bound is not optional: NameResQuery.strings has no upper limit, so an unbounded gather would open one socket per string and could exhaust the process's file descriptors and stampede Solr. Sequential execution used to make that impossible, so the parallelization is what introduces the hazard and the semaphore is what contains it.

The limit applies per request rather than per process, so the queries in flight against Solr is that number multiplied by the number of bulk lookups being served at once. 100 is set deliberately high on the assumption that Solr can take the strain; sizing it against real traffic is being worked out in TranslatorSRI/babel-validation#107, and it is an environment variable so it can be retuned per deployment without a code change.

Two hazards that parallelism sharpened are closed off with it. The limit is clamped to at least 1, since SOLR_MAX_CONCURRENT_LOOKUPS=0 would build a semaphore nobody can acquire and hang every bulk request with no error and no log line. And Solr queries now have a timeout (SOLR_TIMEOUT_SECONDS, default 60) instead of timeout=None: sequentially a stalled connection held up one query, but concurrently it can pin an otherwise-complete bulk request indefinitely while holding a semaphore slot.

The default search is not "fuzzy". Documentation and parameter descriptions called it that; it never was. LowerTextField is a StandardTokenizer and a LowerCaseFilter with no stemming, stopwords, synonyms or ngrams, and lookup() escapes Solr's ~ operator out of the query string, so no caller can request edit-distance matching either. The default search is tokenized: it matches the query's tokens in any order, rewarding order and adjacency with score rather than requiring them, and treats the final token as a prefix when autocomplete=true. Calling it fuzzy invited callers to expect typo tolerance the service has never had.

Documentation. documentation/API.md gained an "Exact matching" section, the exact parameter and the notes above — that file enumerates every /lookup parameter and shows a complete /bulk-lookup body, so omitting exact would have made it wrong rather than merely incomplete. documentation/Deployment.md gained the two new environment variables, the OpenAPI service blurb now mentions exact mode alongside autocomplete, and CLAUDE.md gained a Gotchas section for the three things here that a reader cannot infer from the code.

What it produces

Exact matching is cheaper than the equivalent tokenized search because there is nothing to score: no eDisMax parsing, no phrase or field boosts, just a term lookup against a field holding each name as a single token. Bulk requests of ordinary size now complete in roughly the time of their slowest lookup rather than the sum of all of them.

The exactish filter is marked {!cache=false} on purpose. Solr's filterCache is bounded by entry count (512 in solrconfig.xml), and what it holds are the shared, reusable filters — types:, taxa:, curie: — that nearly every search benefits from. An exactish clause is one distinct entry per distinct search string, so caching it would let a single bulk NER request evict the whole cache and slow down the ordinary search path as collateral damage, for a hit rate near zero on its own entries. Repeated identical exact lookups are still served from the queryResultCache, which caches the whole result and is bounded by RAM.

Because there is no relevance score to rank by in exact mode, results are sorted by clique_identifier_count DESC, curie_suffix ASC instead of the usual score DESC, …. The score field is still present in each result but carries no ranking information.

What it deliberately does not do

• No stemming, normalisation, or punctuation folding beyond lowercasing. Exact means exact; a caller who wants looser matching already has the default search.
• No cap on strings. The semaphore throttles a large bulk request rather than rejecting it, so existing clients sending long lists keep working and simply queue.
• No change to which cliques the default search returns. With exact omitted the generated Solr query is what it was before; the only behavioural difference anywhere is the Solr client's timeout, which used to be unbounded.

Tests

44, all against a live Solr in CI. Beyond the exact-match modes themselves, several encode promises that a change elsewhere could quietly break: that the default search ignores word order but does not tolerate misspellings, that the exactish filter still reaches Solr marked uncached, that exact matching composes with the biolink_type and prefix filters, that the autocomplete/exact rejection survives being raised inside asyncio.gather(), and that a bulk lookup carrying more strings than the concurrency limit still returns every one of them correctly keyed.

The exact-match cases live in tests/test_exact_mode.py, following the repository's existing habit of splitting tests by topic; tests/test_service.py keeps the ones that are about the default search or about bulk lookup generally.

Before merging

Nothing blocking.

Two things to pick up afterwards, both tracked outside this PR because neither can be settled from inside it:

• TranslatorSRI/babel-validation#107 — confirm SOLR_MAX_CONCURRENT_LOOKUPS = 100 against the real request rate, specifically the peak number of concurrent /bulk-lookup requests and the tail of the strings-length distribution, whose product is what Solr actually sees. Needs log analysis, and the value is an environment variable, so it does not gate this.
• NCATSTranslator/Babel#1073 — put each clique's preferred name into its synonyms. If that lands, exact=synonyms and exact=any converge for most cliques, and the three-way split is worth revisiting then rather than pre-emptively now.

History — rebased from the ci branch onto main, then three rounds of review. Kept for anyone tracing why a particular line looks the way it does; the durable conclusions are in the code, the docs and CLAUDE.md above.

This PR originally targeted ci, a long-lived branch that was never merged — its metrics work was reimplemented independently in v1.6.0 rather than ported. Retargeting to main therefore could not be a merge, since that would have dragged forty-odd unrelated ci commits into the diff. Instead the two exact-mode commits were replayed onto main with git rebase --onto.

Anything from ci that this branch used to sit on top of is deliberately gone as a result: the api/solr.py extraction, and the query_log / perf_counter_ns timing. The code here uses main's time.time_ns() timing and main's debug parameter. Do not try to re-port those from ci.

The conflicts were almost entirely a collision of two additions in the same places: main had added a debug parameter exactly where this branch added exact. Both now coexist across /lookup GET and POST, lookup(), and NameResQuery, and main's debug handling sits above the if exact: branch so that the two compose.

The concurrency bound was added after review, not in the original implementation — the first version of the parallelization gathered without any limit, and the limit was 10 before being raised to 100.

A later review round found the filterCache problem, the three silent parameter interactions, and the unvalidated settings; each is one commit. An earlier revision of this PR claimed filter-query caching as the reason exact mode is fast, which was wrong in exactly the workload the feature exists for — the code comment, documentation/API.md and this description all said so until that round.

The exact-match tests were then split out of test_service.py into tests/test_exact_mode.py, which had grown to 487 lines with half its tests belonging to this branch. The exact-plus-filters test failed on its first run because HP:0001300 is typed Disease, not PhenotypicFeature, despite the HP prefix.