feat(components): an artifact that breaks stops breaking in silence (…

[damienesp]

Follows up on #4025. Closes product#4081.

The problem

An artifact built from leadbay_artifact_kit runs in a chat-hosted page whose
only channel out is window.cowork.callMcpTool. Until now the runtime reported
nothing, so every way an artifact could break died in the browser: a dropdown
that populated empty, a button that did nothing when clicked, a call that timed
out or came back as a failure envelope, text that would not parse. None of it
reached us. Artifact-originated tool calls were also indistinguishable from
agent-originated ones on every dashboard.

What this does

The @leadbay/components runtime now reports its own failures, automatically,
through a new leadbay_artifact_event ingest tool. The MCP splits them the way
it already splits its own failures:

Kind	Where it fires	Sink
bridge_unavailable	no window.cowork	Sentry
call_timeout	host never settled	Sentry
call_failed	host rejected, or isError	Sentry
parse_failed	content[0].text would not parse	Sentry
options_empty	picker loaded with zero options	PostHog
action_blocked	validation stopped a run before any call	PostHog
result_rejected	resolved call carried {error:true} / failed checkResult	PostHog

The dividing line is whether something threw. The three outcome kinds are cases
where the call succeeded and the UI is simply wrong — Sentry would never see
them, and they are what a user experiences as "the artifact is broken."

Exceptions go to Sentry under a new ExceptionCtx.source value, "artifact",
fingerprinted ["mcp", "artifact", tool, code]. The error was thrown in the
browser and crossed the bridge as JSON, so there is no stack to forward and
default fingerprinting would collapse every artifact failure into one issue.
These are reports, not stack traces.

Outcomes go to a new mcp artifact event PostHog event carrying only
kind, surface, kit_version, tool, code.

Provenance

A new _origin meta-field, extracted alongside _triggered_by and stripped
before execute(). The runtime stamps "artifact" on every call it makes;
an agent-issued call omits it and records as "agent". It is emitted as
origin on the existing mcp tool called / mcp composite call events, so
artifact traffic separates from agent traffic on the dashboards we already
have rather than needing a parallel event family.

_origin is deliberately NOT folded into _triggered_by: that field is the
auditable record of what the user said, and overwriting it with a provenance
label would destroy the trace it exists to keep. An unrecognized _origin
value falls back to "agent" rather than passing through.

Constraints from the spec

• Not leadbay_report_friction. That tool is consent-gated, carries the
  user's own words, and must never fire unprompted. This one is automatic
  diagnostics and is a separate path end to end. The tool description tells the
  agent explicitly not to call it, and points user-raised problems at
  report_friction instead.
• The leadbay_set_telemetry opt-out is honored with no new plumbing.
  The event is an ordinary tool call, so the existing suppressTelemetry
  predicate already silences it for an opted-out user. Asserted by test.
• No user text, ever. Only bounded enums and error codes travel — never a
  message. LbError.message and envelope message can contain API payload
  text the user never approved for sending (the product#3943 line), so they are
  dropped at the source. Envelope code is a bounded enum and is kept.

Three safety properties in the runtime

1. Never re-entrant. report() uses the raw host bridge, bypassing
   call(), normalize() and withTimeout(). A telemetry call routed through
   call() would, on timeout, emit a timeout event that would itself time out.
2. Fire-and-forget. A rejected emit is swallowed. Telemetry never surfaces
   in a view-model's .error and never blocks a user action.
3. Bounded. Deduped on (kind, surface, tool, code) and capped at 40 events
   per page, so a poll failing every 3s emits once, not thousands of times.

Notes for review

• withSurface restores its ambient marker synchronously, when the loader's
  promise is created, not when it settles. That is why call() snapshots
  runningSurface at entry before its first await rather than reading it in
  its catch — an earlier version read it in the catch and misattributed every
  resource/list failure as surface:"call". Two tests cover this.
• leadbay_artifact_event lives in tools/ (granular-shaped, static relay) so
  it stays out of COMPOSITE_FILE_TOOL_NAMES and carries no _triggered_by
  mandate — an artifact button click has no fresh user utterance to quote. It
  is registered in compositeReadTools so it is always exposed, alongside
  leadbay_artifact_kit; a kit without its sink is a blind artifact.
• Enum values are validated explicitly in execute(), not left to the
  inputSchema: the MCP SDK does not enforce enums before dispatch (same
  reasoning as leadbay_set_telemetry's BAD_ACTION guard), and an unknown
  kind would otherwise become an unbounded Sentry tag / PostHog property.
• @leadbay/components bumped to 0.6.0. kit_version rides on every event
  so a regression pins to a kit release.
• No routing frontmatter on the new template: the tool is machine-called and
  should never be reached from user language, so it is correctly absent from
  TOOLS_WITH_ROUTING.
• WORKFLOWS.md row 59 added.

Testing

All new files, per repo convention. 48 new tests:

• packages/components/test/runtime-telemetry.test.ts (19) — each failure
  point emits, classification is right, no message text escapes, no feedback
  loop, dedupe and cap hold, setTelemetry(false) silences.
• packages/core/test/unit/tools/artifact-event.test.ts (11) — forwards the
  event, makes no API call, rejects unknown kind/surface, bounds field lengths.
• packages/mcp/test/artifact-event-telemetry.test.ts (18) — the Sentry-vs-
  PostHog routing decision, fingerprint separation, the opt-out, and _origin
  (including that it is stripped before execute() and that a spoofed value
  falls back to agent).

pnpm -r typecheck green on all five packages. Components 349/349 and core
1792/1792 green. The components drift guard (build --check) passes.

Pre-existing failures, unrelated to this PR — verified reproducing
identically on a clean stash of this branch, all Windows-environment-specific:
6 in test/audit/{registry-publish-retry-window,release-outcome-alarm}.test.ts,
2 in test/unit/update-state.test.ts (POSIX file modes and symlink creation),
and 1 in promptforge's snippet-references.test.ts (backslash path
separators). Untouched here; worth a separate look on a Windows dev setup.

🤖 Generated with Claude Code

Close leadbay/product#4081

[github-actions]

Couldn't auto-sync with main — this PR has a merge conflict outside the version files. main shipped 0.39.10; please rebase manually.

[claude]

Well-structured feature with strong test coverage for the intended paths (dedupe, cap, no-message, fingerprinting, opt-out via env var, _origin stripping). Two issues found on close reading:

1. Privacy gap on the hosted server — the new captureArtifactEvent method isn't wired into bindTelemetryIdentity (packages/mcp/src/http-server.ts), so it skips the per-user leadbay_set_telemetry opt-out check that every other analytics capture method gets on the multi-tenant HTTP transports. The PR's tests only cover the stdio/env-var suppression path, so this wasn't caught.
2. Telemetry misattribution race — lastTool in packages/components/src/runtime.ts is a shared global read after an await, unlike runningSurface which is deliberately snapshotted before the first await to avoid exactly this. Concurrent field/list loads can cause options_empty events to report the wrong tool.

Neither is CLAUDE.md-guideline-specific (no Tool.ui/generated-file/budget issues found); both are correctness/privacy bugs in the new code itself. Details inline.

[claude]

Both issues from the previous CHANGES_REQUESTED review (5266170056) are resolved in this diff:

1. Hosted opt-out gap — captureArtifactEvent is now wired into bindTelemetryIdentity (packages/mcp/src/http-server.ts), gated by isSuppressed and threaded with identity, matching every other passive capture. New regression test: packages/mcp/test/audit/artifact-event-hosted-optout.test.ts.
2. lastTool cross-attribution race — replaced with a per-load LoadSlot snapshotted synchronously at call() entry (packages/components/src/runtime.ts), mirroring the existing runningSurface discipline. New regression test: packages/components/test/runtime-telemetry-concurrency.test.ts, which exercises interleaved concurrent loads.

No new issues found in this diff. (Note: I was unable to programmatically dismiss the prior CHANGES_REQUESTED review — tooling limitation, not a signal of unresolved concerns.)

fixed