Add a research tool that answers from several sources at once

[devbadya]

Answering a web question took the model three of its four tool rounds — search, choose a result, read it, read another for a second opinion — and in practice it stopped at one source. research folds that chain into a single call, the way weather folds three forecast services.

What it does

One question in. It searches, reorders the results so the first hit on each host comes first, fetches up to five sources in parallel, and returns two quoted passages from each with the URL it came from. Run against the live web:

Researched "Who is the chief executive of Nvidia?" across 5 sources, all read in full.

1. Jensen Huang — https://en.wikipedia.org/wiki/Jensen_Huang
   "As of 2024, Huang has been Nvidia's chief executive for over three decades, a tenure
    described by The Wall Street Journal as "almost unheard of in fast-moving Silicon Valley"."
2. Exec Bios — https://nvidianews.nvidia.com/bios
   "Jensen Huang founded NVIDIA in 1993 and has served since its inception as president,
    chief executive officer and a member of the board of..."
...

Why most of what it reads is thrown away

More of the web reaching the model makes answers worse, not better: function-calling accuracy falls by 7% to 91% as tool responses grow, and five pages at read_page's cap would be 40,000 characters. So pages are read in full and quoted in part, and the whole result is capped at 4,000 characters — five sources for half the context one whole page costs.

Which parts get quoted is decided lexically, like skill retrieval: paragraphs are scored by the question's words, each weighted by how rare it is across everything the call fetched. That makes a stop list unnecessary, since a word in every paragraph earns almost nothing.

The weighting has to be pooled across sources, and the tests pin why. Measured per page, who is the chief executive of the airline ranked the paragraph containing "the airline" exactly level with the one naming the chief executive — within four paragraphs the is as rare as executive. Across the hundred-odd paragraphs five pages produce, the is worth about two per cent of executive. The fixtures are therefore pages rather than snippets, because the mechanism only works at the scale it runs at.

What the live web changed

The first run against real pages returned five genuine sources per question and quoted largely furniture from them. Each failure is now a filter whose test fixture is the string that actually came back — see research_quality_before_after.log for the verbatim before and after.

Quoted instead of the answer	Why it won
These cookies may store a unique ID… — two of five sources	A consent notice is several sentences and names its own site
Wikipedia's reference list, ↑ Retrieved December 24, 2024	Citations repeat the subject once per entry, so they outscore it
Sucuri WebSite Firewall - Access Denied	The reader answers 200, so nothing upstream saw a failure
Sie befinden sich hier … | Startseite	A breadcrumb clears every length floor
Who leads NVIDIA?	A heading is the densest window a paragraph contains

The block page now falls back to the search snippet for the same URL, which turned out to be the best passage in the German run. The breadcrumb needs no word list: prose ends in a full stop and menus do not, in either language. The two word lists have a stated cost — a paragraph genuinely about cookies is dropped with the banners.

Two more failures came from removing markup without leaving a separator, which is the trap unbold already documents for snippets: our@NVIDIATwitter account,NVIDIA Facebookpage was three adjacent links whose brackets were deleted rather than replaced, and [^)]* stopped at the first bracket of Betreuung_(Recht). Bare URLs and footnote anchors go too — reviewAnswer reads every URL in a tool result as a citable source, so a #cite_note anchor in a passage becomes a source that states nothing.

Three things about what a source is worth

• Many sources means many different ones. A news search returns four pages of one newspaper. Results are reordered rather than filtered, so Wikipedia — where every result is one host — refills the list with further articles instead of collapsing to a single source.
• A page that will not open becomes its search snippet, and the header says how many did. The reader's per-minute budget is shared, so a 429 on the fourth source is ordinary rather than a reason to discard the three that arrived.
• Nothing readable throws. Reported as an empty result, a 0.8B model relays it as "there is nothing on this subject", which is the one answer a page that merely failed to load must not produce.

The answer check gained a fourth check

single-source fires when three or more sites were returned and the reply rests on one — without it, the extra sources are reader requests spent on nothing. It stays as shy as the other three: three sites are required, because two is what an ordinary search-then-read turn produces and citing the page you read is correct; it counts hosts, so two pages of one newspaper are one source; and grounding is settled first, so an answer citing nothing is asked for a source before it is asked for a second one.

What this costs

Six of the twenty requests a minute the keyless reader allows — one search plus five pages — so three research questions in a minute is the honest ceiling. That is why this stops at five sources rather than fifteen. LangSearch takes the search off that budget on a free key.

research and web_search both ship, and the difference is cost: one request against six. Which one a turn sees is decided by the skill that routes it — research-question offers only research, lookup-term still searches — rather than by the model weighing it up.

Verification

pnpm check (937 tests) and pnpm build pass. 42 tests cover the tool: host diversity, markdown cleaning, each boilerplate filter against its observed string, pooled passage selection, digest assembly and the whole call over a stubbed network. The review check has 9 more.

The tool was also run against the live web in both English and German — that output is the artifacts in this PR, and is what the filters above were written from. Two eval scenarios were added: one measuring whether the answer actually cites two sites, one that the question survives into the call.

The model itself is not exercised here: there is no GPU in this environment and no CPU fallback, so routing and tool-use rates still need /?eval in Chrome on a real GPU.