Skip to content

test(safe-js): verify each replay checkpoint independently - #720

Closed
kamilio wants to merge 1 commit into
mainfrom
codex/ppr2-deadline
Closed

test(safe-js): verify each replay checkpoint independently#720
kamilio wants to merge 1 commit into
mainfrom
codex/ppr2-deadline

Conversation

@kamilio

@kamilio kamilio commented Sep 10, 2026

Copy link
Copy Markdown
Member

The generator workflow (co) runs seven interpreter executions inside one five-second test. The exact PR719 focused file reproduces its CI timeout. Phase tracing measured 6.78 seconds cumulatively, with each interpreter run around 0.8–1.1 seconds; snapshot capture adds about 20–30 milliseconds.

Test each scenario's public, signal, and completed checkpoint as a complete independent case. Each gets fresh native execution and setup, preserves all capture validations, and verifies its selected replay, recapture, snapshot immutability, receipts, and absence of repeated host effects. Runtime behavior, timeouts, concurrency, and assertions are unchanged.

Validation: the original focused file failed with 18 passes and one five-second timeout. The revised file passes all 29 cases under Node20.20.0 and the maintained SafeJS package configuration; the three co cases took 2184ms, 2099ms, and 2126ms. A bounded profile attributes most execution time to retained-root collection and recursive data measurement; this PR makes no runtime optimization. CI on the new head remains required.

This one-file PR now starts from current main ede5985e5e2b35b38eebd0faaeb3ed6a390493e9, which already contains PR715’s explicit provider models, Kimi registration, built MCP fixture, and Goose catalog fixes. The checkpoint test was unchanged on main. After restoring the updated lockfile dependencies, the rebased change at a9dc74cdce2cebcd3e42c34111efc3e59058cadc passes all 29 focused cases; the three co cases take 1216ms, 1127ms, and 1182ms under Node20.20.0 with the maintained configuration. Fresh combined CI remains required.

The earlier baseline/profile evidence below remains tied to PR719’s source and the original test change; the screenshot is historical evidence at its explicitly labeled commit.

Diagnostic test evidence

Actual saved local baseline, phase-trace, and passing-test excerpts, rendered without rerunning tests. The source/config invariance described above ties this evidence to the one-file change at 8f59f4bdd456fdb478202dc89984ca8466aaac74. This is diagnostic evidence, not product UI or hosted CI clearance.

Diagnostic test evidence — not product UI

@kamilio
kamilio changed the base branch from codex/owned-output-operation-drain-current to main September 10, 2026 23:28
@kamilio
kamilio force-pushed the codex/ppr2-deadline branch from e3114c3 to 8f59f4b Compare September 10, 2026 23:28

@poe-code-agent poe-code-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The co scenario combined seven interpreter executions into one five-second case. Exact PR719 reproduction failed that deadline; phase tracing measured 6.78 seconds, dominated by retained-data accounting during repeated runs.

Parameterize public, signal, and completed checkpoint contracts with independent setup. Preserve native output, pending-boundary, replay, recapture, immutability, receipt, and no-reexecution assertions without changing timeouts or concurrency.

Validation: original focused file reproduced 18 passes and one timeout; revised file passes all 29 cases. The three co cases took 2184ms, 2099ms, and 2126ms under Node20 with the maintained package configuration.
@kamilio
kamilio force-pushed the codex/ppr2-deadline branch from 8f59f4b to a9dc74c Compare September 10, 2026 23:44

@poe-code-agent poe-code-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kamilio

kamilio commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Closing as superseded by the runtime performance fix already on main (024ecd01b), rather than adding repeated setup and interpreter executions.

Fresh review confirmed that each existing checkpoint replay already restores an independent snapshot and uses fresh host/request state; no state-isolation defect was demonstrated. This patch redistributes work across test deadlines and increases interpreter runs from seven to nine per scenario.

Current main was qualified by the full maintained npm test on a46e034059bbe80f84070becaa6f7ff406ee6e05: all 19 original tests in ppr2-integration-adjudication.test.ts passed in 4.346 seconds total; the previously slow co case passed in 1.705 seconds under its unchanged 5-second deadline. The full test run, including SafeJS and posttest, exited zero. Existing assertions and coverage are retained.

The PR was reviewed but its test-overhead tradeoff is no longer needed for the reported timeout.

@kamilio kamilio closed this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant