Skip to content

Add deterministic agent widget capture - #59

Open
SunkenInTime wants to merge 12 commits into
masterfrom
t3code/agent-widget-capture
Open

Add deterministic agent widget capture#59
SunkenInTime wants to merge 12 commits into
masterfrom
t3code/agent-widget-capture

Conversation

@SunkenInTime

@SunkenInTime SunkenInTime commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add weaver capture <widget> --out <file.png> for deterministic, windowless widget execution
  • emit a PNG, semantic snapshot, and machine-readable receipt while isolating registry, host, provider daemon, and user state
  • support explicit clock advancement, semantic actions, provider fixtures, session record/replay, and actionable failure output
  • document the agent workflow and add end-to-end capture smoke coverage

Stack

Depends on Native SDK #28. Merge that PR first, then this one; this branch pins Native commit 05e7e551 and updates the release audit accordingly.

Validation

  • npm test — 85 passed
  • npm run typecheck — passed
  • npm run build — passed
  • npm run audit:release — passed
  • npm run test:capture — passed: pixels, resize, semantics, actions, providers, isolation, failure health, and session replay
  • node cli/bin/weaver.js check examples/noro-signal — passed
  • cd runtime && zig build test -Dplatform=null --summary all — 72 passed, 1 skipped
  • Native zig build test --summary all — 1,840 passed, 2 skipped
  • Native null ReleaseFast build — 6/6 steps passed
  • Native macOS ReleaseFast build — 9/9 steps passed

Receipts

  • runtime binary: 14,238,944 bytes before, 14,326,224 bytes after; +87,280 bytes (+0.613%)
  • capture stdout is exactly one compact JSON receipt line and matches the on-disk receipt
  • session replay produces a byte-identical clock PNG
  • isolated fake-home sentinel remained unchanged across capture
  • an action-induced widget failure now exits nonzero with CaptureWidgetFailed and publishes no artifacts
  • artifact publication fault injection proves rollback restores the prior PNG, snapshot, and receipt even when removal fails; rollback attempts all outputs before reporting an incomplete restore

Noro Signal dogfood

  • added a materially different 430 x 248 Noro media-player example and fixed-input capture coverage
  • confirmed visual and semantic capture: 112 commands, 63 semantic nodes, no warnings, and the expected provider-driven track/time/control labels
  • documented a verified renderer break: a shadowed content stack drops its children; the example uses the documented wrapper workaround
  • drove the PAUSE action and found that an unavailable media transport could publish an unhandled-promise error panel as status: "ok"; capture now rejects it with CaptureWidgetFailed
  • documented the remaining transport-fixture limitation, the unnamed seek-overlay semantic gap, and concrete framework proposals

The screenshots, exact reproduction commands, causal boundaries, and proposals are in the Noro Signal capture evaluation.

Known baseline gate findings

Native scripts/gate.sh fast passes core Zig tests, validation, frontends, and mobile. It still reports pre-existing example API drift plus four machine-sensitive benchmark overruns; this change does not alter those budgets.

Implements the agent widget capture blueprint.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

@SunkenInTime

Copy link
Copy Markdown
Collaborator Author

@greptileai

@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown

Greptile Summary

This change adds windowless widget capture with image, semantic snapshot, receipt, actions, provider fixtures, and replay support for CI and agent workflows. The capture handoff was verified to remove live provider endpoints and artwork-cache inputs, use a fresh state root, and reject provider fixtures that omit subscribed providers or name undeclared providers.

A fixed capture clock still renders different time and calendar fields in different runner timezones. This breaks reproducible screenshots and semantic snapshots for time-based widgets and should be fixed before merge.

T-Rex validation blocked

Native widget rendering could not run because the native runtime executable is absent and the checked-out Native SDK does not provide AppArtifacts.backend required by runtime/build.zig. Native capture should be rerun after restoring the pinned Native SDK revision on macOS or Windows. Configure VMs

Confidence Score: 4/5

Do not merge until fixed-clock capture uses a timezone-independent representation for time-provider fields.

The time discrepancy was reproduced through the bundled SDK reconciliation path using the same fixed epoch under UTC and America/Los_Angeles. The CLI fixture-validation and environment-isolation paths were exercised with before-and-after checks. Native rendering itself remains unavailable in this environment because of the Native SDK mismatch and unsupported host platform.

Files Needing Attention: sdk/src/reconciler.ts needs a capture-aware UTC time conversion. Native capture behavior in runtime/src/main.zig should be rerun after restoring the pinned runtime/native-sdk revision.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex produced proof for a posted P1 finding.
  • Validated the initial runtime setup and documented blocker due to the native weaver-widget executable being absent, with live provider pipe and endpoint inputs in the parent process.
  • Confirmed that the CLI validator and the runtime enforce the same declared-provider and missing-provider constraints.
  • Compared time capture outputs across UTC and America/Los_Angeles, confirming timezone-dependent results and that the runtime reconciliation path uses the provider's actual values.
  • Reviewed CLI and runtime guards for provider fixtures, including non-declared and missing provider handling, with nonzero capture results and explicit errors.

View all artifacts

T-Rex Ran code and verified through T-Rex

Comments Outside Diff (2)

  1. sdk/src/reconciler.ts, line 1136-1138 (link)

    P1 Fixed clocks vary by timezone

    Capture mode fixes an epoch through __weaverCaptureNowMs, but these local-time getters reinterpret that same instant in the runner’s timezone. The fixed instant rendered as 00:30, Monday January 1 in TZ=UTC and 16:30, Sunday December 31 in TZ=America/Los_Angeles. This makes screenshots and semantic time-provider output differ between CI workers. Use UTC getters when the capture clock override is active while retaining local getters for ordinary interactive widgets.

    T-Rex Ran code and verified through T-Rex

  2. General comment

    P1 Capture-mode time display fields depend on local timezone

    • Bug
      • currentTime() uses the fixed __weaverCaptureNowMs epoch but extracts hh, weekday, month, day, and year using local Date getters. The same captured instant renders as 00:30 / Mon Jan 1 in UTC and 16:30 / Sun Dec 31 in America/Los_Angeles.
    • Cause
      • At sdk/src/reconciler.ts:1136-1138, getHours, getDay, getMonth, getDate, and getFullYear interpret the fixed instant in the host process's local timezone.
    • Fix
      • Use the UTC getters for capture-deterministic time fields: getUTCHours(), getUTCMinutes(), getUTCSeconds(), getUTCDay(), getUTCMonth(), getUTCDate(), and getUTCFullYear() (or explicitly specify the intended capture timezone and test it).

    T-Rex Ran code and verified through T-Rex

Reviews (3): Last reviewed commit: "fix: reject degraded captures atomically" | Re-trigger Greptile

@SunkenInTime

Copy link
Copy Markdown
Collaborator Author

@greptileai

Comment thread cli/src/index.ts
Comment thread cli/src/index.ts Outdated
@SunkenInTime

Copy link
Copy Markdown
Collaborator Author

@greptileai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant