FE-1525: Establish the runbook-to-workpiece control and expose the construction gap - #9405
Merged
Conversation
This was referenced Aug 28, 2026
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
15 tasks
lunelson
changed the base branch from
ln/fe-1524-mission-2
to
graphite-base/9405
August 31, 2026 13:00
lunelson
force-pushed
the
graphite-base/9405
branch
from
August 31, 2026 13:00
7616004 to
5b5c8ed
Compare
lunelson
force-pushed
the
ln/fe-1525-headless-runbook-pn
branch
from
August 31, 2026 13:00
b738aa1 to
4bc7ff3
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9405 +/- ##
=======================================
Coverage 60.89% 60.89%
=======================================
Files 1461 1461
Lines 146725 146725
Branches 6744 6744
=======================================
+ Hits 89342 89344 +2
+ Misses 56265 56263 -2
Partials 1118 1118 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
4 tasks
… and evidence One ChatAgent, one runbook skill (defineSkill + on-disk Markdown), headless createFlueClient drive with artifact recovery from history(). Hermetic through line green (IR recovered, parseSDCPNFile ok); two real-model runs recorded with construct rejection documented (weight-0 exclusive arcs, malformed types/ parameters). MISSION.md stays live; acceptance is a human call. SIDE_QUEST.md opens the validated-construction follow-up (Petrinaut-owned contracts, minimal tool subset, reuse of run 2's IR). HANDOFF.md carries the full session state.
Package the modelling skill through Flue, prove bounded headless construction, and record the real-model schema bridge failure for Mission 5. Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Lu Nelson <ln@hash.ai>
Signed-off-by: Lu Nelson <ln@hash.ai>
Keep the reviewed hermetic-entry-point inventory aligned with the runbook artifact, faux-provider, and headless integration tests added by Mission 3.
Replace the requested-scope prune exception with explicit core test task dependencies on the app and shipped plugin builds. Task-aware affected selection and pruning now carry those workspaces through the normal graph.
The app's tests read committed runbook evidence and inbox examples outside any workspace, so Turbo pruning omitted them unless the app scope copies those paths explicitly.
The only production caller recovers the fixed runbook-ir fence, so remove the unused dynamic language regex instead of accepting a pattern-bearing input.
lunelson
force-pushed
the
ln/fe-1525-headless-runbook-pn
branch
from
September 3, 2026 14:27
da93cba to
174d74a
Compare
lunelson
had a problem deploying
to
pull-request
September 3, 2026 14:28 — with
GitHub Actions
Failure
kostandinang
approved these changes
Sep 3, 2026
TimDiekmann
approved these changes
Sep 3, 2026
TimDiekmann
left a comment
Member
There was a problem hiding this comment.
Thank you for sorting the tasks!
infra ✅
Contributor
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🌟 What is the purpose of this PR?
Brunch could elicit a useful runbook through its production Flue agent, but its construction proof treated parser acceptance as success even when the resulting Petri net was empty. That made the evidence look stronger than the behavior and left reviewers without a trustworthy control for later runbook redesign.
This PR establishes the runbook-to-workpiece control, proves non-empty hermetic construction, and preserves the real-model construction failure as design evidence. Elicitation and workpiece recovery are established; semantic real-model Petri-net construction remains explicitly unproven and moves to successor work.
🔗 Related links
docs/mission-archive/3-structurally-typed-runbook-to-headless-pn.mddocs/evidence/implementations/fe-1525-headless-runbook-pn.mddocs/evidence/evaluations/vestera-prospective-baseline-v1/campaign-adjudication.mddocs/specs/structurally-typed-elicitation-runbooks.md🚫 Blocked by
main; required CI and review remain.🔍 What does this change?
The implementation follows one production throughline from skill activation and interview turns to a recoverable workpiece and bounded construction. The changes below are the concrete review surface.
This branch also intentionally retires the completed issue/PR legibility migration package during documentation weeding. That package is unrelated to FE-1525 behavior, has no live consumer, and remains recoverable from the durable
mainancestorf897a699c55c9e41666fe1dbaf2db74fd85a4546; it is not being restored or replaced with a live-tree tombstone here.sdcpn-modellingskill on the productionChatAgent, with a concise router and four packaged Markdown resources for elicitation, workpiece structure, PN construction, and checks.createFlueClient → send → wait → history()drive through the built application and recovers the finalrunbook-irworkpiece from canonical Flue history./api/chatconversations.main, preserving the merged idempotency, history-projection, build-cache, and CI-pruning fixes.REQUESTED_DEPENDENCIESprune exception is removed.🏗️ Agent notes
Branch mission
Imperative: Prove that one production Flue
ChatAgentcan use a structurally typed runbook to conduct or replay an interview, maintain a structured Markdown workpiece, and use it to construct Petri-net JSON Petrinaut accepts. Recover the useful teaching from the slow typed-capture path without restoring that kernel or joining Mission 2's capture ledger.Throughline:
createFlueClient → initial modelling request → ChatAgent activates sdcpn-modelling and reads phase resources → driver send/wait/history across interview turns → recover runbook-ir → construct-from-IR request → read construction/check resources → validated mutation tools or PN output → Petrinaut parse/validate.Proof: One mounted skill and concise router; packaged resources readable from the built app; universal and SDCPN teaching separated from scenario facts; expert-vocabulary interviewing; recoverable Markdown workpiece with unknowns, assumptions, conflicts, and omissions; construction consuming that workpiece; losses named; no interviewer sweep tool or capture-store write. This proof does not establish a final heading catalogue, typed capture/IR, panel canvas tools, session-as-net, two brains, or an automated repertoire compiler.
Constraints: Use Flue's skill/resource and JS-client paths rather than a custom loader or runner; one agent and one skill; keep elicitation and construction in separate information regions; keep reusable resources at target-formalism level; do not restore closed kinds/slots/fold/completion machinery; do not join the workpiece to the capture store; keep construction tools off ordinary panel conversations; use Petrinaut's canonical validation contracts rather than copied payload types.
Fog-line: Exact heading catalogue; resource split; activation router; workpiece recovery shape; construction-discovered return to elicitation; sufficiency of
parseSDCPNFile; universal-versus-SDCPN editorial migration; production packaging; model and latency behavior.Stop or reorient: Stop if the system prompt absorbs the runbook, another agent/skill/loader/workflow appears without observed pressure, construction vocabulary leaks into ordinary interviewing, scenario facts enter reusable teaching, the workpiece requires transcript rereading, the typed kernel or capture join returns, canvas tools reach the interviewer, or ordinary teaching turns return to minute-scale latency.
Observed proof
addType.elementscalls were rejected, no net elements were constructed, and the resulting empty file passedparseSDCPNFileonly vacuously. Proof item 6 therefore remains open for semantic fidelity.Fog-line answers
sdcpn-modellingplus a six-line router activated reliably in real runs.runbook-irfence in assistant history is a sufficient recovery seam; no new persistence surface was needed.parseSDCPNFileis strict enough for file shape but insufficient as sole evidence of semantic construction; non-empty and semantic inspection are required.defineSkillpackaging failed from the built app. A bare staticSKILL.mdimport packages the complete directory and works fromdist/app.mjs.Reparent and CI reconciliation
The 15 Mission 3 commits were replayed onto current
main. The only rebase conflicts were in the emitted-bundle assertion and Petrinaut-chat integration test. Resolution retainedmain's precise route assertions, idempotency/result-signal accounting, and pending-client-tool projection while preserving Mission 3's built-app loader andappTransportcapture-history path.Tim Diekmann's late observation on #9463 is carried forward:
@hashintel/brunch-agent#test:unitnow declares task dependencies on the app and both plugin builds. Main's task-awareaffectedTasksselection andprune-scopes.shtherefore retain these test fixtures through the normal Turbo graph. The bespoke requested-scope exception and its special-case tests are removed.FE-1563 remains untouched and is intentionally marked as needing restack until its local work is ready.
Carried flags
Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
The affected Brunch workspaces are private.
📜 Does this require a change to the docs?
The changes in this PR:
The mission archive, implementation proof, runbook specification, evaluation protocols, evidence, research synthesis, and successor concerns are updated together.
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect this🐾 Next steps
🛡 What tests cover this?
apps/brunch-agent/test/runbook-headless.test.tsandrunbook-headless.integration.tsdrive the built application through the packaged skill and headless client.apps/brunch-agent/test/petrinaut-construction-tools.test.tsandheadless-petrinaut-client.test.tscover canonical mutation validation and client callback behavior.apps/brunch-agent/test/petrinaut-chat.test.tskeeps the ordinary panel path free of construction tools while preserving the merged retry, history, ownership, and capture-sweep proofs.Focused verification passes for
@apps/brunch-agentand@hashintel/brunch-agent: build dependencies, unit tests, Oxlint, and TypeScript. The prune-action Python suite passes 5/5 tests.❓ How to test this?
mise exec -- turbo run test:unit lint:eslint lint:tsc --env-mode=loose --filter '@apps/brunch-agent' --filter '@hashintel/brunch-agent'from the HASH root.cd .github/actions/prune-repository && python3 -m unittest prune_test.py..github/scripts/prune-scopes.sh '@hashintel/brunch-agent'under the repository's mise environment and confirm it includes the app, binding, transport, both plugins, and core.apps/brunch-agent/test/runbook-headless.test.tsmust activate the packaged skill, read its resources, recover the workpiece, execute validated construction, and parse a non-empty net.📹 Demo
No new recording accompanies the reparent. The production-path tests and committed run/evaluation artifacts are the durable proof; the paid construction failure is intentionally retained as evidence rather than replayed.