Check that replies copy the researched Answer line

.cursor/skills/debug-model-output/SKILL.md

@@ -60,7 +60,8 @@ Three things follow from that, and all three are easy to undo by accident:
   A check that cannot point at a tool result does not belong here.
 - **A check that fires on a correct answer is a bug**, not a strict setting. It costs a generation
   and trains the user to ignore the label. `review.test.ts` pins the shy cases — a clarifying
-  question, a rounded decimal, a source carried over from an earlier turn — and they are the point.
+  question, a rounded decimal, a researched surname standing in for the full name, a source
+  carried over from an earlier turn — and they are the point.
 - **Only successful tool results become evidence.** A failed fetch has nothing to check against, and
   demanding a citation for a page that never loaded is worse than saying nothing.
 

README.md

@@ -550,19 +550,20 @@ Every reply says which skill answered it and how it was found: `weather skill ·
 
 A skill fires on some requests. This runs on all of them.
 
-Between the model settling on an answer and that answer reaching the screen, `src/agent/review.ts` reads it back against what the turn actually produced — the results the tools returned, and the URLs already in the conversation. Three things are checked:
+Between the model settling on an answer and that answer reaching the screen, `src/agent/review.ts` reads it back against what the turn actually produced — the results the tools returned, and the URLs already in the conversation. Four things are checked:
 
-| Check             | Fires when                                                            |
-| ----------------- | --------------------------------------------------------------------- |
-| `wrong-number`    | The calculator's value, or the clock's local HH:MM, is stated nowhere |
-| `invented-source` | The answer cites a URL that no tool returned and nobody supplied      |
-| `missing-source`  | Tools returned sources and the answer cites none                      |
+| Check             | Fires when                                                                   |
+| ----------------- | ---------------------------------------------------------------------------- |
+| `wrong-number`    | The calculator's value, or the clock's local HH:MM, is stated nowhere        |
+| `wrong-fact`      | `research` opened with `Answer: …` and that name or figure is stated nowhere |
+| `invented-source` | The answer cites a URL that no tool returned and nobody supplied             |
+| `missing-source`  | Tools returned sources and the answer cites none                             |
 
 A failed check costs one further generation. The model is handed its own draft and told what to change — _The calculator returned 6748 \* 9 = 60732. Give that number, exactly as it came back._ — and the correction replaces the draft only if it leaves fewer problems behind. Otherwise the draft stands. That gate is the important half: the correction comes from the same 0.8B model, so a mechanism that could not tell an improvement from a regression would be a coin toss on every reply.
 
 **The checks are deterministic, and that is the design.** Asking the model to grade its own answer spends exactly the capacity the answer needed, and intrinsic self-correction — re-reading with nothing new to go on — degrades reasoning rather than improving it ([arXiv:2310.01798](https://arxiv.org/html/2310.01798)). What works is external feedback, so every check compares the draft against something already in the context, and the correction states the fix rather than inviting the model to hunt for one.
 
-They are also deliberately shy. A clarifying question is asked for no citation; a long decimal quoted to fewer places counts as the calculator's number; citing the site when a page on it was read is close enough; a URL from an earlier reply is not an invention; a year-only clock answer is left alone, and a German date like `27.08.2026` is not a time. Every check would rather miss a mistake than invent one, because a check that fires on a correct answer costs a generation and teaches you to ignore the whole mechanism.
+They are also deliberately shy. A clarifying question is asked for no citation; a long decimal quoted to fewer places counts as the calculator's number; a researched surname (`Merz`) counts as the full extract (`Friedrich Merz`); 14 million counts as 13.96 million, which is the same reading the extractor already accepted; citing the site when a page on it was read is close enough; a URL from an earlier reply is not an invention; a year-only clock answer is left alone, and a German date like `27.08.2026` is not a time. Every check would rather miss a mistake than invent one, because a check that fires on a correct answer costs a generation and teaches you to ignore the whole mechanism.
 
 The interface says what happened rather than quietly rewriting the reply. While the corrected answer streams in it is labelled with what is being fixed, and afterwards it carries `corrected` — claimed only for an answer that now passes every check — or `flagged`, naming what is still wrong with the text on screen. An answer half fixed and advertised as corrected would be worse than no check at all.
 

src/agent/budget.test.ts

@@ -93,4 +93,26 @@ describe('budgetFallback', () => {
     expect(text).toContain('Try narrowing the question')
     expect(splitSources(text).sources).toEqual([])
   })
+
+  it('hands over the researched one-liner when the wind-down came back empty', () => {
+    const text = budgetFallback(
+      evidence([
+        {
+          tool: 'research',
+          result: [
+            'Answer: Friedrich Merz.',
+            '',
+            'Researched 2026-09-10 for "Bundeskanzler" across 1 source, all read in full.',
+            '',
+            '1. Bundeskanzler — https://de.wikipedia.org/wiki/Bundeskanzler',
+            '   "Amtsträger ist Friedrich Merz."',
+          ].join('\n'),
+        },
+      ]),
+    )
+
+    expect(text).toMatch(/^Friedrich Merz\./)
+    expect(text).not.toContain('could not settle')
+    expect(splitSources(text).sources).toEqual(['https://de.wikipedia.org/wiki/Bundeskanzler'])
+  })
 })

src/agent/budget.ts

@@ -1,5 +1,5 @@
 import { MAX_TOOL_ROUNDS } from '@/llm/config'
-import { findUrls, type ReviewEvidence } from './review'
+import { findUrls, researchedAnswer, type ReviewEvidence } from './review'
 
 /**
  * What happens when a turn runs out of tool rounds.
@@ -117,12 +117,19 @@ const FALLBACK_SOURCES = 3
  * into citation pills — the pages are the part worth clicking.
  */
 export function budgetFallback(evidence: ReviewEvidence): string {
-  const opening = `I could not settle on an answer within ${MAX_TOOL_ROUNDS} rounds of tool calls.`
+  const extracted = researchedAnswer(evidence)
   const sources = [...new Set(evidence.toolResults.flatMap(({ result }) => findUrls(result)))].slice(
     0,
     FALLBACK_SOURCES,
   )
 
+  // `research` already committed to a one-liner. Handing that over beats an
+  // apology: the wind-down round failed, not the search.
+  if (extracted) {
+    return sources.length > 0 ? `${extracted}.\n\nSource: ${sources.join(' ')}` : `${extracted}.`
+  }
+
+  const opening = `I could not settle on an answer within ${MAX_TOOL_ROUNDS} rounds of tool calls.`
   if (sources.length === 0) return `${opening} Try narrowing the question.`
   return `${opening} These pages came up on the way, in case one of them helps.\n\nSource: ${sources.join(' ')}`
 }

src/agent/loop.test.ts

@@ -315,6 +315,29 @@ describe('checking the answer before returning it', () => {
     )
   })
 
+  it('corrects a researched name the answer dropped', async () => {
+    const research = defineTool('research', 'research', { type: 'object', properties: {} }, async () =>
+      [
+        'Answer: Ama Osei.',
+        '',
+        'Researched 2026-09-10 for "who runs Fictional Airways" across 1 source, all read in full.',
+        '',
+        '1. Leadership — https://fictionalairways.example/leadership',
+        '   "Ama Osei has led the airline since 2023."',
+      ].join('\n'),
+    )
+    const client = fakeClient([
+      toolCall('research', 'query', 'who runs Fictional Airways'),
+      'guessing</think>Piet Hendriks runs it.\n\nSource: https://fictionalairways.example/leadership',
+      'reading the digest</think>Ama Osei.\n\nSource: https://fictionalairways.example/leadership',
+    ])
+
+    const result = await runAgent(client, turns, [research], callbacks())
+
+    expect(result.content).toContain('Ama Osei')
+    expect(result.review).toEqual({ found: ['wrong-fact'], corrected: true })
+  })
+
   it('corrects a number the answer did not take from the calculator', async () => {
     const client = fakeClient([
       toolCall('calculator', 'expression', '6748 * 9'),

src/agent/review.test.ts

@@ -1,5 +1,11 @@
 import { describe, expect, it } from 'vitest'
-import { collectEvidence, correctionPrompt, reviewAnswer, type ReviewEvidence } from './review'
+import {
+  collectEvidence,
+  correctionPrompt,
+  researchedAnswer,
+  reviewAnswer,
+  type ReviewEvidence,
+} from './review'
 
 function evidence(overrides: Partial<ReviewEvidence> = {}): ReviewEvidence {
   return { toolResults: [], knownUrls: [], ...overrides }
@@ -223,6 +229,143 @@ describe('reviewAnswer', () => {
       ])
     })
   })
+
+  describe('researched facts', () => {
+    const researched = evidence({
+      toolResults: [
+        {
+          tool: 'research',
+          result: [
+            'Answer: Friedrich Merz.',
+            '',
+            'Researched 2026-09-10 for "Bundeskanzler" across 2 sources, all read in full.',
+            '',
+            '1. Bundeskanzler — https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)',
+            '   "Amtsträger ist seit dem 6. Mai 2025 Friedrich Merz (CDU)."',
+          ].join('\n'),
+        },
+      ],
+    })
+
+    it('accepts the name the research digest opened with', () => {
+      expect(
+        checks(
+          'Friedrich Merz, seit Mai 2025.\n\nSource: https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)',
+          researched,
+        ),
+      ).toEqual([])
+    })
+
+    it('accepts the surname when the full name was extracted', () => {
+      // A check that flagged "Merz" would fire on a correct German short answer.
+      expect(
+        checks('Merz.\n\nSource: https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)', researched),
+      ).toEqual([])
+    })
+
+    it('accepts a missing accent on a copied name', () => {
+      const un = evidence({
+        toolResults: [
+          {
+            tool: 'research',
+            result:
+              'Answer: António Guterres.\n\nResearched 2026-09-10 for "UN" across 1 source, all read in full.\n\n1. UN — https://www.un.org/sg/en',
+          },
+        ],
+      })
+
+      expect(checks('Antonio Guterres.\n\nSource: https://www.un.org/sg/en', un)).toEqual([])
+    })
+
+    it('catches an invented name after research already answered', () => {
+      expect(
+        checks(
+          'Olaf Scholz.\n\nSource: https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)',
+          researched,
+        ),
+      ).toEqual(['wrong-fact'])
+    })
+
+    it('catches a reply that quotes the office and drops the incumbent', () => {
+      expect(
+        checks(
+          'The chancellor is the head of government.\n\nSource: https://de.wikipedia.org/wiki/Bundeskanzler_(Deutschland)',
+          researched,
+        ),
+      ).toEqual(['wrong-fact'])
+    })
+
+    it('quotes the extract in the correction', () => {
+      const [finding] = reviewAnswer('Olaf Scholz.', researched)
+
+      expect(finding?.instruction).toContain('Answer: Friedrich Merz')
+    })
+
+    it('accepts a figure the digest opened with, including German decimals', () => {
+      const population = evidence({
+        toolResults: [
+          {
+            tool: 'research',
+            result:
+              'Answer: 13.96 million.\n\nResearched 2026-09-10 for "population of Tokyo" across 1 source, all read in full.\n\n1. Tokyo — https://en.wikipedia.org/wiki/Tokyo',
+          },
+        ],
+      })
+
+      expect(
+        checks('About 13,96 Millionen.\n\nSource: https://en.wikipedia.org/wiki/Tokyo', population),
+      ).toEqual([])
+      expect(checks('About 14 million.\n\nSource: https://en.wikipedia.org/wiki/Tokyo', population)).toEqual(
+        [],
+      )
+    })
+
+    it('catches an invented figure after research already answered', () => {
+      const population = evidence({
+        toolResults: [
+          {
+            tool: 'research',
+            result:
+              'Answer: 13.96 million.\n\nResearched 2026-09-10 for "population of Tokyo" across 1 source, all read in full.\n\n1. Tokyo — https://en.wikipedia.org/wiki/Tokyo',
+          },
+        ],
+      })
+
+      expect(
+        checks('Tokyo has 11 million people.\n\nSource: https://en.wikipedia.org/wiki/Tokyo', population),
+      ).toEqual(['wrong-fact'])
+    })
+
+    it('leaves a biography digest alone when nothing was extracted', () => {
+      const bio = evidence({
+        toolResults: [
+          {
+            tool: 'research',
+            result:
+              'Researched 2026-09-10 for "Who is Elon Musk" across 1 source, all read in full.\n\n1. Elon Musk — https://en.wikipedia.org/wiki/Elon_Musk\n   "Elon Musk is a businessman."',
+          },
+        ],
+      })
+
+      expect(
+        checks('Elon Musk is a businessman.\n\nSource: https://en.wikipedia.org/wiki/Elon_Musk', bio),
+      ).toEqual([])
+    })
+
+    it('does not treat an Answer line inside a web_search snippet as an extract', () => {
+      const searched = evidence({
+        toolResults: [
+          {
+            tool: 'web_search',
+            result: '1. Quiz\n   https://quiz.example\n   Answer: Paris is a cheese.',
+          },
+        ],
+      })
+
+      expect(researchedAnswer(searched)).toBeNull()
+      expect(checks('Lyon.\n\nSource: https://quiz.example', searched)).toEqual([])
+    })
+  })
 })
 
 describe('collectEvidence', () => {