Skip to content

feat: build_pathway over MCP, and a richer widget-to-agent channel - #76

Open
adamstankiewicz wants to merge 6 commits into
mainfrom
feat/mcp-pathway-and-reporting
Open

feat: build_pathway over MCP, and a richer widget-to-agent channel#76
adamstankiewicz wants to merge 6 commits into
mainfrom
feat/mcp-pathway-and-reporting

Conversation

@adamstankiewicz

@adamstankiewicz adamstankiewicz commented Aug 30, 2026

Copy link
Copy Markdown
Owner

What & why

Pathways reach MCP hosts two ways, matching how agents actually work: build_pathway runs the full pipeline as one tool call (~30s), persists the session under a minted learner id, and returns the /learn/{sessionId} student link plus a structured plan summary — honest when storage or verification is absent, with the ownerId returned because the edit endpoint gates on it. The connector instructions also teach the conversational mode explicitly: the agent is the sequencer (show_widget → read evidence → next activity).

The evidence channel back to the agent gains two things: completion reports carry a structured widget_result block alongside the prose sentence (the shape the universal WidgetResult converges on), and a single early struggle signal after consecutive wrong checks with no visible progress — so an agent can help before the finish line, not only after it.

The tools take an audience, not a grade. gradeHint forced every caller to describe a learner as a K-12 student; a deployment against ABET criteria, a cert blueprint or an internal competency model had nothing honest to put there, and the argument is free text steering standard selection — nothing about it needs to be a grade. build_pathway is new here so it is named right at birth; show_widget ships on main, so it gains audience as the documented argument and keeps gradeHint as an accepted alias.

Type of change

  • Contract change (new tool + a structured model-context reporting convention)

Breaking or contract changes

Additive: one new tool; ui/update-model-context messages gain an optional fenced-JSON second content block (size-capped, over-fenced so payload text can't escape into prose). Existing single-block consumers are unaffected.

show_widget's input schema changes — additively, but on a tool that is already deployed: it gains audience and retains gradeHint, which now documents itself as a deprecated alias. Callers written against the shipped tool keep working unchanged; the alias is pinned by a test so a later cleanup can't silently drop it. Worth a reviewer's eye since this is the one contract here that third parties already program against.

How it was verified

  • pnpm lint and rm -rf .next && npx tsc --noEmit
  • Tool surface pinned through tools/list with real Requests: the three tools expose audience, the two new ones never carried the old name, and show_widget's alias is asserted
  • pnpm mcp:build and committed the shell
  • Review-pass findings fixed and dispositioned (see comment): honest save failures, ownerId returned, actual-vs-planned widget kinds, progress-aware struggle heuristic, per-widget counter reset, fence safety
  • One manual connector test before merge: point Claude at a deployment, call build_pathway, open the returned link — the only live path not yet exercised end-to-end

Merge note: #75 has merged, so that conflict is resolved; this branch carries the audience rename forward from it. Still rebases after #86 (which drops this branch's committed-shell blob — the shell regenerates from source).

Authorship

  • I understand every line of this diff
  • Agent-assisted or agent-authored: authored by Claude working interactively with @adamstankiewicz; multi-agent review findings + dispositions in the PR comment
  • No package.json / pnpm-lock.yaml changes

🤖 Generated with Claude Code

@vercel

vercel Bot commented Aug 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
interactive-learning-experiences Ready Ready Preview Aug 31, 2026 12:39am

Request Review

@adamstankiewicz

Copy link
Copy Markdown
Owner Author

Pre-merge review pass (multi-agent finders + verification) — findings and dispositions, so the review here can focus on the deltas:

Fixed in 179e7a3:

  • Persistence failures no longer discard a 30s build behind a wrong "no persistent storage" message — save is separately guarded and the real reason is returned with the intact plan summary.
  • The minted ownerId is now returned in structuredContent (the edit endpoint gates on it; discarding it made MCP-built pathways permanently uneditable by anyone).
  • Per-step widgetKind reports the kind actually built; the planned kind + substitution note ride along when generation fell back — the caller can no longer describe widgets that aren't in the pathway.
  • Struggle signal: consecutive-wrong streak with a progress heuristic (multi-part widgets like crossword emit correct:false while advancing — that's progress, not struggle); counters reset when the host hands the frame a new spec.
  • Model-context detail blocks are size-capped and over-fenced (payload text containing ``` can't break into prose); dedupe includes the structured detail.
  • runPathway passes through teacherNote/lessonPlanExcerpt (the assign route carries a third copy of this collect loop and can now converge), collects substitution notes, and drops a dead in-band error branch.

Deferred, deliberately:

  • AbortSignal threading through streamPathway (disconnected clients still bill the full pipeline) — pre-existing, shared with /api/pathway and /api/assign; wants one repo-wide fix.
  • Converging /api/assign onto runPathway — mechanical follow-up, not this PR.
  • The progress heuristic's durable replacement is completion-semantics metadata on the registry entry (registry-owned-semantics refactor).

🤖 Generated with Claude Code

@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 179e7a3 to 5a1e55b Compare August 30, 2026 18:31
@adamstankiewicz
adamstankiewicz changed the base branch from main to feat/a2ui-agui-conformance August 30, 2026 18:49
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 5a1e55b to 3fdbb0f Compare August 30, 2026 18:49
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 3fdbb0f to 81d8ecc Compare August 30, 2026 18:55
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 81d8ecc to d2f874f Compare August 30, 2026 18:56
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from d2f874f to 8fa8581 Compare August 30, 2026 19:06
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 4d56e1a to bb9300f Compare August 30, 2026 19:14
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from bb9300f to a7c6969 Compare August 30, 2026 19:28
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 20ea1ad to 1f5fbb0 Compare August 30, 2026 21:08
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 1f5fbb0 to ee83337 Compare August 30, 2026 21:13
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from ee83337 to 599f25c Compare August 30, 2026 21:19
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 599f25c to bded23d Compare August 30, 2026 21:25
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from bded23d to defc0e6 Compare August 30, 2026 21:27
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from defc0e6 to b3df0ab Compare August 30, 2026 21:29
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from b3df0ab to 1336458 Compare August 30, 2026 21:32
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 1336458 to 87e0e09 Compare August 30, 2026 21:40
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 87e0e09 to 269a502 Compare August 30, 2026 21:44
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from 269a502 to c38077a Compare August 30, 2026 23:31
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from c38077a to e508ac8 Compare August 30, 2026 23:41
@adamstankiewicz
adamstankiewicz force-pushed the feat/mcp-pathway-and-reporting branch from e508ac8 to 66434d2 Compare August 30, 2026 23:52
adamstankiewicz and others added 6 commits August 30, 2026 20:18
Pathways reach MCP hosts two ways now, matching how agents actually work:

- build_pathway: the full pipeline as one tool call (~30s, maxDuration
  raised) — plans 4-6 sequenced activities against a verified standard,
  persists the session under a minted learner id, and returns the
  /learn/{sessionId} student link plus a structured plan summary. With no
  persistent storage it says so honestly instead of inventing a link.
- The connector instructions now teach the other way explicitly: in a
  live conversation the agent IS the sequencer — show_widget, read the
  evidence, choose the next activity.

The evidence channel back to the agent gets two upgrades:

- Completion reports now carry a structured widget_result block (kind,
  standard, correct, attempts, hints, score, detail) alongside the prose
  sentence — the shape the SDK's universal WidgetResult converges on —
  via updateModelContext growing an optional detail rider (host bridge
  and the in-widget reportToConversation both).
- One early struggle signal: after three wrong checks without completion,
  the conversation hears about stuck work once, so an agent can help
  before the finish line instead of only after it.

Shell rebuilt (public/widget-shell.html).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From the pre-merge review pass:

- build_pathway: persistence is separately guarded so a storage failure
  returns the built pathway with the real reason instead of discarding
  five model calls behind a wrong 'no persistent storage' guess; the
  minted ownerId is returned (the edit endpoint gates on it — discarding
  it made MCP-built pathways permanently uneditable); per-step widgetKind
  now reports the kind actually built, with the planned kind and the
  substitution note carried when generation fell back.
- runPathway: passes through teacherNote/lessonPlanExcerpt so the assign
  route's third copy of the collect loop can converge on it; collects
  step notes; drops the dead in-band error branch (the generator throws).
- Struggle signal: consecutive-wrong streak with a progress heuristic
  (solved/placed/matched counters reset it), so a student steadily
  solving a crossword is not reported as stuck; counters reset when the
  host hands the frame a new spec, so widget B never inherits widget A's
  attempts or its already-reported silence.
- Model-context detail blocks: size-capped and fenced with more backticks
  than the content contains, so payload text can't break into prose; the
  dedupe key includes the detail, not just the sentence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…velope

- report-to-host: completion reports once with the structured result;
  struggle fires exactly once after three consecutive wrongs; steady
  multi-part progress (the crossword case that review caught) never
  triggers it; a right answer resets the streak; reset() gives a reused
  frame fresh counters. The fence logic moves to a shared pure helper
  (src/lib/mcp/fence.ts) used by both reporters — tested for escape
  resistance and the size cap.
- runPathway: collects anchor/plan/widgets/notes/rejections from a
  mocked pipeline, throws on a hollow run, passes teacher context
  through — the collector contract the MCP handler leans on.
- /api/mcp protocol envelope: initialize, the four-tool listing with
  MCP Apps metadata, resource listing, unknown method/tool codes, parse
  errors, and build_pathway's topic guard — pinned with real Requests.
  ('server-only' stubbed in vitest resolve; it has no runtime outside
  Next.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`gradeHint` forces every caller to describe a learner as a K-12 student.
A deployment against ABET criteria, a cert blueprint or an internal
competency model has nothing honest to put there, and the argument is free
text steering standard selection — nothing about it needs to be a grade.

`find_activity` and `build_pathway` are new in this stack, so they are named
right at birth and never carry `gradeHint`. `show_widget` ships on main, so
it gains `audience` as the documented argument and keeps `gradeHint` as an
accepted alias — removing it would break callers written against the
deployed tool for no benefit.

The boundary stays honest in both directions: `audience` here is free text
and explicitly a hint, while the scheme-scoped `audience` on an emitted
manifest comes from the graph. The comment at the find_activity call site
says so, because the two fields now share a name and only one is verified.

Tested through tools/list with real Requests, per the wire-surface rule:
the three tools expose `audience`, the two new ones do not carry the old
name, and show_widget's alias is pinned so a later cleanup cannot silently
drop it.

No mcp:build: a server route is not reachable from widget components.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Naming the tool input `audience` reintroduced exactly the ambiguity the
rename set out to kill. On a manifest, `audience` is scheme-scoped and
graph-derived — a verified statement about the activity. As a tool input it
is unverified caller text that steers standard proposal. Same word, two
meanings, one surface: a consumer reading both cannot tell which guarantee
it is holding.

The old `gradeHint` was right about this and I dropped the wrong half of it.
The `…Hint` suffix carries the epistemics; only "grade" was the K-12
assumption. So: `audienceHint`.

`show_widget` still accepts `gradeHint` as the alias it shipped with, now
pointing at `audienceHint`. The tools/list test asserts the new name *and*
that no tool exposes a bare `audience`, so the collision cannot come back by
someone reaching for the shorter word.

No behavior change: same argument, same plumbing, different name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adam's point: freestyle sequencing throws away exactly what the planner
provides — the activate → model → practice → check arc against a
verified standard is the pedagogy, not an artifact of the handoff form.
The connector guidance now says so: in conversation the agent delivers
that arc through show_widget, letting evidence set pacing inside it
(and may call build_pathway first to use its plan as the spine);
build_pathway remains the handoff form for a link.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant