From c601a84ae293e21b8ccdaabb860b54d6ba64c7b1 Mon Sep 17 00:00:00 2001 From: Kostandin Angjellari Date: Thu, 3 Sep 2026 17:09:43 +0200 Subject: [PATCH 1/5] Add Petrinaut Voice Loom script Co-authored-by: Cursor --- ...9-03-petrinaut-voice-loom-script-design.md | 173 ++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md diff --git a/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md b/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md new file mode 100644 index 00000000000..ee0330d246e --- /dev/null +++ b/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md @@ -0,0 +1,173 @@ +# Petrinaut Voice Loom script + +## Purpose + +Show product and design stakeholders how Voice can make a complex Petrinaut +workflow easier to start. The demo uses the Site 1000 raw-material purchasing +model as context, but it is a Voice UX story rather than a tour of the model. + +Target runtime: 4 minutes 15 seconds, with 30–45 seconds of contingency. + +## Simplified domain story + +Keep only the facts needed to understand one decision: + +- Site 1000 makes one product, Sonic Flow. +- Production needs two raw materials from three suppliers. +- Buying too little delays production and customer orders. +- Buying too much raises holding and expiry costs. +- A successful policy delivers at least 95% of demand, keeps production delay + low, and holds each material's expiry rate below 5%. +- The useful experiment compares the baseline purchasing policy with fragile + supply over 104 simulated weeks. + +Do not explain the complexity classification, formulas, token schemas, +place/transition inventories, calibration tables, or full scenario catalogue. +Those details remain available for inspection after Voice has established the +user's intent. + +## Demo preparation + +- Open the purchasing model with the AI assistant closed. +- Use a fresh browser profile if the consent disclosure must be shown. +- Start from a stable interview question such as: “Walk me through the process + you want to model.” +- Confirm the microphone and Voice preview configuration before recording. +- Keep the model canvas visible behind the assistant panel. +- If the assistant varies the wording of a question, preserve the intent of + the scripted answer rather than trying to reproduce every word exactly. + +## Recording plan + +- **0:00–0:30 — Frame the problem.** Show the model and explain that the + underlying process is rich, but a user should not need to structure all of + it before getting started. +- **0:30–1:00 — Enter Voice.** Open the assistant, select Voice, acknowledge + the disclosure, and let the status move from Connecting to Listening. +- **1:00–2:05 — Describe the process.** Answer the first interview question in + ordinary language. Point out the provisional transcript and finalized + answer's Voice provenance without pausing the flow. +- **2:05–2:55 — State the decision and constraints.** Let the assistant ask a + narrower follow-up. Explain the purchasing trade-off and the three success + criteria. +- **2:55–3:35 — Define one experiment.** Ask to compare the baseline policy + against fragile supply over two years. +- **3:35–3:55 — Show recoverability.** Briefly identify mute, repeat question, + read full response, and end controls. Do not demonstrate every control. +- **3:55–4:15 — Close on the UX value.** Emphasize progressive disclosure: + speak first, inspect the transcript, then refine the structured model. + +## Presenter bullet points + +- The problem is not a lack of model power; it is the cost of expressing domain + knowledge in a structured form. +- Voice starts with the user's language and narrows the problem one question at + a time. +- The UI makes system state explicit: Connecting, Listening, Thinking, and + Speaking. +- Spoken answers become visible, inspectable conversation turns with Voice + provenance. +- The user stays in control with mute, playback, transcript, and end controls. +- The demo ends with one concrete experiment, not a complete explanation of the + net. + +## Full transcript + +**[0:00 — Model canvas visible]** + +“This is a raw-material purchasing model for a pharmaceutical factory. The +full model includes demand variation, supplier outages, shipment delays, +quality checks, production, stock expiry, and cost. + +That level of detail is useful, but it creates a difficult starting point. The +Voice experience is intended to let a domain expert begin with the process they +already understand, while Petrinaut progressively structures it.” + +**[0:30 — Open the assistant and select Voice]** + +“I’ll open the assistant and switch from text to Voice. + +Before the microphone starts, Petrinaut explains that OpenAI processes the live +audio, while Petrinaut keeps the finalized answers in the conversation rather +than storing the audio. I can check the microphone, give consent, and start.” + +**[Start Voice; let Connecting change to Listening]** + +“The dock makes the current state clear. It connects, then tells me when it is +listening.” + +**Assistant, expected question:** “Walk me through the process you want to +model.” + +**Presenter, speaking to Voice:** + +“Site 1000 produces one finished product called Sonic Flow. It needs two raw +materials: Sonaflozin and Flowbind. Sonaflozin comes from one supplier, and +Flowbind can come from either a primary supplier or a smaller backup supplier. + +Materials are prepared, shipped, quarantined, and quality released before +production. The main risk is that disruption or delay leaves production +without material, while excess stock costs money and may expire.” + +**[Pause while the answer is finalized]** + +“My words appear immediately as provisional transcription, then settle into a +normal conversation turn. The waveform marker shows that this answer came from +Voice. I can inspect the exact text instead of relying on an invisible audio +history.” + +**Assistant, expected follow-up:** “What decision should the model help you +make?” + +**Presenter, speaking to Voice:** + +“Help us decide when to reorder each material and how much to buy. We want the +lowest total cost without creating shortages or excess inventory.” + +**Assistant, expected follow-up:** “How will you judge whether a policy is +acceptable?” + +**Presenter, speaking to Voice:** + +“Evaluate it over 104 weeks. It must fill at least 95 percent of customer +demand, keep average production delay at or below a quarter of a week per +planned unit, and keep the expiry rate of each raw material below 5 percent.” + +**[Let the assistant respond; keep the Voice dock visible]** + +“Notice that the conversation has moved from a broad process description to a +decision and measurable constraints. I did not need to begin with formulas, +token types, or a list of forty-five transitions.” + +**Assistant, expected follow-up:** “What would you like to test first?” + +**Presenter, speaking to Voice:** + +“Compare the current reorder points and target levels with the same policy +under fragile supply, where supplier outages are more frequent. Show the effect +on total policy cost, fill rate, holding cost, late-delivery penalties, and +supplier switching.” + +**[Point briefly to the dock controls]** + +“The dock stays compact while the session is active. I can mute the microphone, +show or hide transcription, repeat the current question, hear the full +response, or end Voice without leaving the model.” + +**[End Voice; leave finalized conversation visible]** + +“The intended UX is progressive disclosure: start by speaking naturally, +preserve the result as inspectable text, and then refine the structured model +and experiment visually. + +For this inventory example, Voice turns a very large specification into one +clear task: find a purchasing policy that protects customer service without +paying for too much stock.” + +## Review criteria + +- The spoken content fits comfortably inside five minutes. +- Voice remains the main subject throughout. +- The model is understandable without specialist Petri-net terminology. +- Every demonstrated control or status exists in the current Voice UI. +- The closing experiment is concrete enough to continue in Petrinaut. From ab98f3d27e4dce202a84d8df77d13f021eb0d14f Mon Sep 17 00:00:00 2001 From: Kostandin Angjellari Date: Thu, 3 Sep 2026 17:35:02 +0200 Subject: [PATCH 2/5] Expose current Petrinaut net context to Brunch Co-authored-by: Cursor --- apps/brunch-agent/src/agents/chat-agent.ts | 7 ++ apps/brunch-agent/src/client-tool.ts | 6 +- .../src/tools/get-latest-net-definition.ts | 22 ++++ .../test/petrinaut-chat-result.ts | 7 ++ .../test/petrinaut-chat.integration.ts | 117 ++++++++++++++++-- apps/brunch-agent/test/petrinaut-chat.test.ts | 42 +++++-- 6 files changed, 184 insertions(+), 17 deletions(-) create mode 100644 apps/brunch-agent/src/tools/get-latest-net-definition.ts diff --git a/apps/brunch-agent/src/agents/chat-agent.ts b/apps/brunch-agent/src/agents/chat-agent.ts index de58d5c3b19..ba77d9a862a 100644 --- a/apps/brunch-agent/src/agents/chat-agent.ts +++ b/apps/brunch-agent/src/agents/chat-agent.ts @@ -11,6 +11,10 @@ import { defineSkill, useModel, useSkill, useTool } from "@flue/runtime"; import { ASK_TOOL_NAME } from "@hashintel/brunch-agent/client-tools"; import { brunchAsk } from "../tools/brunch-ask.ts"; +import { + getLatestNetDefinition, + GET_LATEST_NET_DEFINITION_TOOL_NAME, +} from "../tools/get-latest-net-definition.ts"; import { ping } from "../tools/ping.ts"; import { readPetrinautDoc } from "../tools/read-petrinaut-doc.ts"; @@ -33,12 +37,15 @@ export function ChatAgent() { useModel(`anthropic/${CHAT_MODEL_ID}`); useSkill(confirmPath); useTool(ping); + useTool(getLatestNetDefinition); useTool(readPetrinautDoc); useTool(brunchAsk); return [ "You are a concise assistant inside the Petrinaut editor.", "Call ping when you need to confirm the server tool path.", `Activate the \`${STUB_SKILL_NAME}\` skill before calling ping.`, + `Before answering any request about this net, the current net, or the existing net—including before beginning an interview—call \`${GET_LATEST_NET_DEFINITION_TOOL_NAME}\`.`, + `Do not say the canvas is unavailable while you can call \`${GET_LATEST_NET_DEFINITION_TOOL_NAME}\`.`, "When the user asks how Petrinaut's UI works, call readPetrinautDoc.", "A client-tool-result signal is JSON [{ toolCallId, toolName, output }]. Treat output as the browser's result for that call and continue helping the user.", `When the user explicitly requests an interview, call \`${ASK_TOOL_NAME}\`.`, diff --git a/apps/brunch-agent/src/client-tool.ts b/apps/brunch-agent/src/client-tool.ts index 4161e8c27bc..7b9b8c73cb1 100644 --- a/apps/brunch-agent/src/client-tool.ts +++ b/apps/brunch-agent/src/client-tool.ts @@ -1,7 +1,10 @@ /** Flue-side client-tool signal contract: awaiting sentinel, result signal, tool names. */ import { ASK_TOOL_NAME } from "@hashintel/brunch-agent/client-tools"; -import { readPetrinautDocToolName } from "@hashintel/petrinaut-core/ai"; +import { + getLatestNetDefinitionToolName, + readPetrinautDocToolName, +} from "@hashintel/petrinaut-core/ai"; export const CLIENT_TOOL_RESULT_SIGNAL = "client-tool-result"; @@ -9,6 +12,7 @@ export const AWAITING_CLIENT = "client" as const; export const clientToolNames: ReadonlySet = new Set([ ASK_TOOL_NAME, + getLatestNetDefinitionToolName, readPetrinautDocToolName, ]); diff --git a/apps/brunch-agent/src/tools/get-latest-net-definition.ts b/apps/brunch-agent/src/tools/get-latest-net-definition.ts new file mode 100644 index 00000000000..08bae192ba4 --- /dev/null +++ b/apps/brunch-agent/src/tools/get-latest-net-definition.ts @@ -0,0 +1,22 @@ +import { defineTool } from "@flue/runtime"; +import * as v from "valibot"; + +import { getLatestNetDefinitionToolName } from "@hashintel/petrinaut-core/ai"; + +import { AWAITING_CLIENT } from "../client-tool.ts"; + +export const GET_LATEST_NET_DEFINITION_TOOL_NAME = + getLatestNetDefinitionToolName; + +export const getLatestNetDefinition = defineTool({ + name: GET_LATEST_NET_DEFINITION_TOOL_NAME, + description: + "Get the live Petrinaut net state as `{ title, definition, extensions }`. The browser executes this tool. After you call it, wait for a client-tool-result signal carrying the current state, then continue from that state.", + input: v.strictObject({}), + output: v.object({ + awaiting: v.literal(AWAITING_CLIENT), + }), + run() { + return { output: { awaiting: AWAITING_CLIENT }, terminate: true }; + }, +}); diff --git a/apps/brunch-agent/test/petrinaut-chat-result.ts b/apps/brunch-agent/test/petrinaut-chat-result.ts index 683105e4ac3..95df6a45287 100644 --- a/apps/brunch-agent/test/petrinaut-chat-result.ts +++ b/apps/brunch-agent/test/petrinaut-chat-result.ts @@ -16,10 +16,17 @@ export interface PetrinautChatResult { { type: "tool-input-available" } > | null; readonly clientToolOutputsOnInitial: readonly UIMessageChunk[]; + readonly latestNetDefinitionCall: Extract< + UIMessageChunk, + { type: "tool-input-available" } + > | null; + readonly latestNetDefinitionOutputsOnInitial: readonly UIMessageChunk[]; readonly initialFinish: UIMessageChunk | undefined; readonly pendingHistoryClientToolState: string | undefined; + readonly pendingHistoryLatestNetDefinitionState: string | undefined; readonly resumedStatus: number; readonly resumedText: string; + readonly resumedTextBeforeAsk: string; readonly resumedFinish: UIMessageChunk | undefined; readonly askCall: Extract< UIMessageChunk, diff --git a/apps/brunch-agent/test/petrinaut-chat.integration.ts b/apps/brunch-agent/test/petrinaut-chat.integration.ts index 44333dbeabf..e6f9528baf8 100644 --- a/apps/brunch-agent/test/petrinaut-chat.integration.ts +++ b/apps/brunch-agent/test/petrinaut-chat.integration.ts @@ -14,6 +14,7 @@ import { sqlite, start } from "@flue/runtime/node"; import { createFlueClient, FlueApiError } from "@flue/sdk"; import { ASK_TOOL_NAME } from "@hashintel/brunch-agent/client-tools"; +import { getLatestNetDefinitionToolName } from "@hashintel/petrinaut-core/ai"; import { ACTIVATE_SKILL_TOOL_NAME, @@ -42,6 +43,58 @@ const principalKey = "principal-mission-1"; const conversationId = "conversation-mission-1"; const identity = { principalKey, conversationId }; const instanceId = flueConversationIdFrom(identity); +const latestNetDefinitionFixture = { + title: "Invoice review conveyor", + definition: { + places: [ + { + id: "incoming-invoices", + name: "Incoming invoices", + colorId: null, + dynamicsEnabled: false, + differentialEquationId: null, + x: 100, + y: 100, + }, + { + id: "approved-invoices", + name: "Approved invoices", + colorId: null, + dynamicsEnabled: false, + differentialEquationId: null, + x: 500, + y: 100, + }, + ], + transitions: [ + { + id: "review-invoice", + name: "Review invoice", + inputArcs: [ + { placeId: "incoming-invoices", weight: 1, type: "standard" }, + ], + outputArcs: [{ placeId: "approved-invoices", weight: 1 }], + lambdaType: "predicate", + lambdaCode: "return true;", + transitionKernelCode: "", + x: 300, + y: 125, + }, + ], + types: [], + parameters: [], + differentialEquations: [], + subnets: [], + componentInstances: [], + }, + extensions: { + colors: false, + stochasticity: false, + dynamics: false, + parameters: false, + subnets: false, + }, +} as const; const dbPath = process.env.BRUNCH_CHAT_DB_PATH ?? (await mkdtemp(join(tmpdir(), "brunch-chat-"))); @@ -142,7 +195,14 @@ try { ), fauxAssistantMessage( [ - fauxThinking("The ping returned. Read the user guide next."), + fauxThinking( + "The ping returned. Read the current net and user guide next.", + ), + fauxToolCall( + getLatestNetDefinitionToolName, + {}, + { id: "tool-net-1" }, + ), fauxToolCall( READ_PETRINAUT_DOC_TOOL_NAME, { doc: "ai-assistant" }, @@ -153,13 +213,18 @@ try { ), fauxAssistantMessage( [ - fauxThinking("The user explicitly requested an interview."), + fauxThinking( + "Use the current net context before the first interview question.", + ), fauxText( - "The guide says the assistant can read its own documentation pages.", + "The Invoice review conveyor moves Incoming invoices through Review invoice into Approved invoices. The guide says the assistant can read its own documentation pages.", ), fauxToolCall( ASK_TOOL_NAME, - { question: "What outcome should this process reliably produce?" }, + { + question: + "What should happen when review cannot approve an invoice?", + }, { id: "tool-ask-1" }, ), ], @@ -168,7 +233,7 @@ try { fauxAssistantMessage([ fauxThinking("Use the correlated client-tool answer."), fauxText( - "I received your answer: a reliable handoff. The Petrinaut canvas was not modified.", + "I received your answer: send it to manual review. The Petrinaut canvas was not modified.", ), ]), fauxAssistantMessage([ @@ -199,7 +264,7 @@ try { userMessage.parts = [ { type: "text", - text: "Start an interview and run the FE-1435 transport probe.", + text: "Interview this Petri net. What does it do?", }, ]; @@ -245,6 +310,14 @@ try { chunk.type === "tool-input-available" && chunk.toolName === READ_PETRINAUT_DOC_TOOL_NAME, ) ?? null; + const latestNetDefinitionCall = + initialChunks.find( + ( + chunk, + ): chunk is Extract => + chunk.type === "tool-input-available" && + chunk.toolName === getLatestNetDefinitionToolName, + ) ?? null; const pendingHistoryResponse = await app.fetch( new Request( @@ -263,6 +336,11 @@ try { const pendingHistoryClientToolState = pendingHistoryBody.messages ?.flatMap((message) => message.parts ?? []) .find((part) => part.toolCallId === clientToolCall?.toolCallId)?.state; + const pendingHistoryLatestNetDefinitionState = pendingHistoryBody.messages + ?.flatMap((message) => message.parts ?? []) + .find( + (part) => part.toolCallId === latestNetDefinitionCall?.toolCallId, + )?.state; const resumeBody = { id: conversationId, @@ -274,6 +352,13 @@ try { id: startChunk?.messageId, role: "assistant", parts: [ + { + type: `tool-${getLatestNetDefinitionToolName}`, + toolCallId: latestNetDefinitionCall?.toolCallId, + state: "output-available", + input: {}, + output: latestNetDefinitionFixture, + }, { type: `tool-${READ_PETRINAUT_DOC_TOOL_NAME}`, toolCallId: clientToolCall?.toolCallId, @@ -306,6 +391,16 @@ try { chunk.type === "tool-input-available" && chunk.toolName === ASK_TOOL_NAME, ) ?? null; + const askChunkIndex = resumedChunks.findIndex( + (chunk) => + chunk.type === "tool-input-available" && + chunk.toolName === ASK_TOOL_NAME, + ); + const resumedTextBeforeAsk = resumedChunks + .slice(0, askChunkIndex === -1 ? resumedChunks.length : askChunkIndex) + .filter((chunk) => chunk.type === "text-delta") + .map((chunk) => chunk.delta) + .join(""); const pendingAskHistoryResponse = await app.fetch( new Request( `http://brunch.test/api/chat?id=${encodeURIComponent(conversationId)}`, @@ -339,7 +434,7 @@ try { toolCallId: askCall?.toolCallId, state: "output-available", input: askCall?.input, - output: { answer: "A reliable handoff." }, + output: { answer: "Send it to manual review." }, }, ], }, @@ -483,13 +578,21 @@ try { chunk.type === "tool-output-available" && chunk.toolCallId === clientToolCall?.toolCallId, ), + latestNetDefinitionCall, + latestNetDefinitionOutputsOnInitial: initialChunks.filter( + (chunk) => + chunk.type === "tool-output-available" && + chunk.toolCallId === latestNetDefinitionCall?.toolCallId, + ), initialFinish: initialChunks.at(-1), pendingHistoryClientToolState, + pendingHistoryLatestNetDefinitionState, resumedStatus: resumeResponse.status, resumedText: resumedChunks .filter((chunk) => chunk.type === "text-delta") .map((chunk) => chunk.delta) .join(""), + resumedTextBeforeAsk, resumedFinish: resumedChunks.at(-1), askCall, askToolOutputsBeforeResume: resumedChunks.filter( diff --git a/apps/brunch-agent/test/petrinaut-chat.test.ts b/apps/brunch-agent/test/petrinaut-chat.test.ts index a28ac036c37..b44fc3035c3 100644 --- a/apps/brunch-agent/test/petrinaut-chat.test.ts +++ b/apps/brunch-agent/test/petrinaut-chat.test.ts @@ -69,20 +69,39 @@ test("the committed /api/chat door streams a plain Flue agent through server and expect(result.clientToolCall).not.toHaveProperty("dynamic"); expect(result.clientToolCall).not.toHaveProperty("providerExecuted"); expect(result.clientToolOutputsOnInitial).toEqual([]); + expect(result.latestNetDefinitionCall).toMatchObject({ + type: "tool-input-available", + toolName: "getLatestNetDefinition", + input: {}, + }); + expect(result.latestNetDefinitionCall).not.toHaveProperty("dynamic"); + expect(result.latestNetDefinitionCall).not.toHaveProperty( + "providerExecuted", + ); + expect(result.latestNetDefinitionOutputsOnInitial).toEqual([]); expect(result.initialFinish).toEqual({ type: "finish", finishReason: "tool-calls", }); expect(result.pendingHistoryClientToolState).toBe("input-available"); + expect(result.pendingHistoryLatestNetDefinitionState).toBe( + "input-available", + ); expect(result.resumedStatus).toBe(200); - expect(result.resumedText).toContain( + expect(result.resumedTextBeforeAsk).toContain("Invoice review conveyor"); + expect(result.resumedTextBeforeAsk).toContain("Incoming invoices"); + expect(result.resumedTextBeforeAsk).toContain("Review invoice"); + expect(result.resumedTextBeforeAsk).toContain("Approved invoices"); + expect(result.resumedTextBeforeAsk).toContain( "The guide says the assistant can read its own documentation pages.", ); expect(result.askCall).toMatchObject({ type: "tool-input-available", toolName: "brunch_ask", - input: { question: "What outcome should this process reliably produce?" }, + input: { + question: "What should happen when review cannot approve an invoice?", + }, dynamic: true, }); expect(result.askCall).not.toHaveProperty("providerExecuted"); @@ -94,7 +113,7 @@ test("the committed /api/chat door streams a plain Flue agent through server and expect(result.pendingHistoryAskState).toBe("input-available"); expect(result.answerResumeStatus).toBe(200); expect(result.answerResumeText).toContain( - "I received your answer: a reliable handoff.", + "I received your answer: send it to manual review.", ); expect(result.answerResumeFinish).toEqual({ type: "finish", @@ -107,20 +126,23 @@ test("the committed /api/chat door streams a plain Flue agent through server and expect(result.historyGetStatus).toBe(200); expect(result.historyUserText).toContain( - "Start an interview and run the FE-1435 transport probe.", + "Interview this Petri net. What does it do?", ); + expect(result.historyUserText).not.toContain("Invoice review conveyor"); expect(result.foreignHistoryMessages).toBe(0); expect(result.unauthenticatedHistoryStatus).toBe(401); expect(result.foreignAgentHistoryStatus).toBe(403); expect(result.transcript).toContain( - "Start an interview and run the FE-1435 transport probe.", + "Interview this Petri net. What does it do?", ); expect(result.transcript).toContain("Checking the server, then the docs."); expect(result.transcript).toContain("tool ping"); + expect(result.transcript).toContain("tool getLatestNetDefinition"); + expect(result.transcript).toContain('"title":"Invoice review conveyor"'); expect(result.transcript).toContain("tool readPetrinautDoc"); expect(result.transcript).toContain("tool brunch_ask"); expect(result.transcript).toContain( - '"output":{"answer":"A reliable handoff."}', + '"output":{"answer":"Send it to manual review."}', ); expect(result.transcript).toContain("tool activate_skill"); expect(result.transcript).toContain( @@ -134,18 +156,19 @@ test("the committed /api/chat door streams a plain Flue agent through server and expect(result.interviewerToolNames).toEqual([ "activate_skill", "ping", + "getLatestNetDefinition", "readPetrinautDoc", "brunch_ask", ]); expect(result.captureIds.length).toBe(1); expect(result.captureExcerpts).toEqual([ - "Start an interview and run the FE-1435 transport probe.", + "Interview this Petri net. What does it do?", ]); expect(result.capturePayloads).toEqual([{}]); expect(result.recaptureIds).toEqual(result.captureIds); expect(result.skippedDedupKeys.length).toBeGreaterThan(0); expect(result.captureUserText).toContain( - "Start an interview and run the FE-1435 transport probe.", + "Interview this Petri net. What does it do?", ); expect(inspectionLines[0]).toMatchObject({ @@ -206,9 +229,10 @@ test("the committed /api/chat door streams a plain Flue agent through server and ) as PetrinautResumeResult; expect(resumeResult.historyGetStatus).toBe(200); expect(resumeResult.historyUserText).toContain( - "Start an interview and run the FE-1435 transport probe.", + "Interview this Petri net. What does it do?", ); expect(resumeResult.transcript).toContain("tool ping"); + expect(resumeResult.transcript).toContain("tool getLatestNetDefinition"); expect(resumeResult.transcript).toContain("tool readPetrinautDoc"); expect(resumeResult.transcript).toContain("tool brunch_ask"); expect(resumeResult.transcript).toContain("tool activate_skill"); From a0502cd39875261c8f71321342c1f8eefd8e0423 Mon Sep 17 00:00:00 2001 From: Kostandin Angjellari Date: Thu, 3 Sep 2026 17:37:47 +0200 Subject: [PATCH 3/5] Use camelCase for net client tool name Co-authored-by: Cursor --- apps/brunch-agent/src/agents/chat-agent.ts | 6 +++--- apps/brunch-agent/src/tools/get-latest-net-definition.ts | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/apps/brunch-agent/src/agents/chat-agent.ts b/apps/brunch-agent/src/agents/chat-agent.ts index ba77d9a862a..97020cb18b5 100644 --- a/apps/brunch-agent/src/agents/chat-agent.ts +++ b/apps/brunch-agent/src/agents/chat-agent.ts @@ -13,7 +13,7 @@ import { ASK_TOOL_NAME } from "@hashintel/brunch-agent/client-tools"; import { brunchAsk } from "../tools/brunch-ask.ts"; import { getLatestNetDefinition, - GET_LATEST_NET_DEFINITION_TOOL_NAME, + getLatestNetDefinitionClientToolName, } from "../tools/get-latest-net-definition.ts"; import { ping } from "../tools/ping.ts"; import { readPetrinautDoc } from "../tools/read-petrinaut-doc.ts"; @@ -44,8 +44,8 @@ export function ChatAgent() { "You are a concise assistant inside the Petrinaut editor.", "Call ping when you need to confirm the server tool path.", `Activate the \`${STUB_SKILL_NAME}\` skill before calling ping.`, - `Before answering any request about this net, the current net, or the existing net—including before beginning an interview—call \`${GET_LATEST_NET_DEFINITION_TOOL_NAME}\`.`, - `Do not say the canvas is unavailable while you can call \`${GET_LATEST_NET_DEFINITION_TOOL_NAME}\`.`, + `Before answering any request about this net, the current net, or the existing net—including before beginning an interview—call \`${getLatestNetDefinitionClientToolName}\`.`, + `Do not say the canvas is unavailable while you can call \`${getLatestNetDefinitionClientToolName}\`.`, "When the user asks how Petrinaut's UI works, call readPetrinautDoc.", "A client-tool-result signal is JSON [{ toolCallId, toolName, output }]. Treat output as the browser's result for that call and continue helping the user.", `When the user explicitly requests an interview, call \`${ASK_TOOL_NAME}\`.`, diff --git a/apps/brunch-agent/src/tools/get-latest-net-definition.ts b/apps/brunch-agent/src/tools/get-latest-net-definition.ts index 08bae192ba4..3955dcba3c2 100644 --- a/apps/brunch-agent/src/tools/get-latest-net-definition.ts +++ b/apps/brunch-agent/src/tools/get-latest-net-definition.ts @@ -5,11 +5,11 @@ import { getLatestNetDefinitionToolName } from "@hashintel/petrinaut-core/ai"; import { AWAITING_CLIENT } from "../client-tool.ts"; -export const GET_LATEST_NET_DEFINITION_TOOL_NAME = +export const getLatestNetDefinitionClientToolName = getLatestNetDefinitionToolName; export const getLatestNetDefinition = defineTool({ - name: GET_LATEST_NET_DEFINITION_TOOL_NAME, + name: getLatestNetDefinitionClientToolName, description: "Get the live Petrinaut net state as `{ title, definition, extensions }`. The browser executes this tool. After you call it, wait for a client-tool-result signal carrying the current state, then continue from that state.", input: v.strictObject({}), From 2ab2e400bb61d24d3fc65ae2847cb814b348934a Mon Sep 17 00:00:00 2001 From: Kostandin Angjellari Date: Thu, 3 Sep 2026 17:40:16 +0200 Subject: [PATCH 4/5] Make Petrinaut Voice demo reproducible Co-authored-by: Cursor --- ...9-03-petrinaut-voice-loom-script-design.md | 70 ++++++++++++------- 1 file changed, 44 insertions(+), 26 deletions(-) diff --git a/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md b/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md index ee0330d246e..a5b9fb1b6a9 100644 --- a/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md +++ b/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md @@ -28,10 +28,12 @@ user's intent. ## Demo preparation -- Open the purchasing model with the AI assistant closed. +- Preload or import the Site 1000 purchasing model, then open it with the AI + assistant closed. - Use a fresh browser profile if the consent disclosure must be shown. -- Start from a stable interview question such as: “Walk me through the process - you want to model.” +- Type the kickoff exactly as: `Interview this Petri net. What does it do?` +- Let Brunch inspect the live net and ask its first relevant interview question + before starting Voice. - Confirm the microphone and Voice preview configuration before recording. - Keep the model canvas visible behind the assistant panel. - If the assistant varies the wording of a question, preserve the intent of @@ -42,11 +44,14 @@ user's intent. - **0:00–0:30 — Frame the problem.** Show the model and explain that the underlying process is rich, but a user should not need to structure all of it before getting started. -- **0:30–1:00 — Enter Voice.** Open the assistant, select Voice, acknowledge - the disclosure, and let the status move from Connecting to Listening. -- **1:00–2:05 — Describe the process.** Answer the first interview question in - ordinary language. Point out the provisional transcript and finalized - answer's Voice provenance without pausing the flow. +- **0:30–1:00 — Ground the interview.** Open the assistant, type the exact + kickoff, and let Brunch inspect the live net and ask its first relevant + interview question. +- **1:00–1:20 — Enter Voice.** Select Voice, acknowledge the disclosure, and + let the status move from Connecting to Listening. +- **1:20–2:05 — Describe the process.** Answer the first interview question in + ordinary language. Once the answer is finalized, point out its visible text + and Voice provenance without pausing the flow. - **2:05–2:55 — State the decision and constraints.** Let the assistant ask a narrower follow-up. Explain the purchasing trade-off and the three success criteria. @@ -55,14 +60,15 @@ user's intent. - **3:35–3:55 — Show recoverability.** Briefly identify mute, repeat question, read full response, and end controls. Do not demonstrate every control. - **3:55–4:15 — Close on the UX value.** Emphasize progressive disclosure: - speak first, inspect the transcript, then refine the structured model. + ground the interview in the live net, speak naturally, inspect the finalized + answer, then refine the structured model. ## Presenter bullet points - The problem is not a lack of model power; it is the cost of expressing domain knowledge in a structured form. -- Voice starts with the user's language and narrows the problem one question at - a time. +- After Brunch grounds the interview in the live net, Voice captures the user's + language and narrows the problem one question at a time. - The UI makes system state explicit: Connecting, Listening, Thinking, and Speaking. - Spoken answers become visible, inspectable conversation turns with Voice @@ -83,22 +89,35 @@ That level of detail is useful, but it creates a difficult starting point. The Voice experience is intended to let a domain expert begin with the process they already understand, while Petrinaut progressively structures it.” -**[0:30 — Open the assistant and select Voice]** +**[0:30 — Open the assistant and type the kickoff]** -“I’ll open the assistant and switch from text to Voice. +“I’ll start in text with one exact request: ‘Interview this Petri net. What +does it do?’ This gives Brunch a chance to inspect the live Site 1000 net and +ask a question that is relevant to what is already on the canvas.” -Before the microphone starts, Petrinaut explains that OpenAI processes the live -audio, while Petrinaut keeps the finalized answers in the conversation rather -than storing the audio. I can check the microphone, give consent, and start.” +**Presenter, typing:** `Interview this Petri net. What does it do?` + +**[Wait for Brunch to inspect the live net and ask its first relevant +interview question]** + +**Assistant, expected question:** “Walk me through the process you want to +model.” + +**[1:00 — Select Voice]** + +“Now that Brunch has inspected the model and asked the first interview +question, I’ll switch to Voice. + +Before the microphone starts, Petrinaut explains that OpenAI processes the +live audio, while Petrinaut keeps the finalized answers in the conversation +rather than storing the audio. I can check the microphone, give consent, and +start.” **[Start Voice; let Connecting change to Listening]** “The dock makes the current state clear. It connects, then tells me when it is listening.” -**Assistant, expected question:** “Walk me through the process you want to -model.” - **Presenter, speaking to Voice:** “Site 1000 produces one finished product called Sonic Flow. It needs two raw @@ -111,10 +130,9 @@ without material, while excess stock costs money and may expire.” **[Pause while the answer is finalized]** -“My words appear immediately as provisional transcription, then settle into a -normal conversation turn. The waveform marker shows that this answer came from -Voice. I can inspect the exact text instead of relying on an invisible audio -history.” +“Once my answer is finalized, it appears as a normal conversation turn. The +waveform marker shows that this answer came from Voice. I can inspect the exact +text instead of relying on an invisible audio history.” **Assistant, expected follow-up:** “What decision should the model help you make?” @@ -156,9 +174,9 @@ response, or end Voice without leaving the model.” **[End Voice; leave finalized conversation visible]** -“The intended UX is progressive disclosure: start by speaking naturally, -preserve the result as inspectable text, and then refine the structured model -and experiment visually. +“The intended UX is progressive disclosure: ground the interview in the live +net, speak naturally, preserve the finalized answer as inspectable text, and +then refine the structured model and experiment visually. For this inventory example, Voice turns a very large specification into one clear task: find a purchasing policy that protects customer service without From f4476e62bbb4509ad19a4ced62a8d8fa00d16826 Mon Sep 17 00:00:00 2001 From: Kostandin Angjellari Date: Thu, 3 Sep 2026 17:43:57 +0200 Subject: [PATCH 5/5] Correct Petrinaut Voice demo sequence Co-authored-by: Cursor --- ...9-03-petrinaut-voice-loom-script-design.md | 42 ++++++++++++------- 1 file changed, 27 insertions(+), 15 deletions(-) diff --git a/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md b/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md index a5b9fb1b6a9..2acd20ec2bc 100644 --- a/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md +++ b/docs/superpowers/specs/2026-09-03-petrinaut-voice-loom-script-design.md @@ -6,7 +6,7 @@ Show product and design stakeholders how Voice can make a complex Petrinaut workflow easier to start. The demo uses the Site 1000 raw-material purchasing model as context, but it is a Voice UX story rather than a tour of the model. -Target runtime: 4 minutes 15 seconds, with 30–45 seconds of contingency. +Target runtime: 4 minutes 40 seconds, with 15 seconds of contingency. ## Simplified domain story @@ -44,22 +44,23 @@ user's intent. - **0:00–0:30 — Frame the problem.** Show the model and explain that the underlying process is rich, but a user should not need to structure all of it before getting started. -- **0:30–1:00 — Ground the interview.** Open the assistant, type the exact +- **0:30–1:15 — Ground the interview.** Open the assistant, type the exact kickoff, and let Brunch inspect the live net and ask its first relevant interview question. -- **1:00–1:20 — Enter Voice.** Select Voice, acknowledge the disclosure, and - let the status move from Connecting to Listening. -- **1:20–2:05 — Describe the process.** Answer the first interview question in +- **1:15–1:55 — Enter Voice.** Select Voice, acknowledge the disclosure, let + the status move from Connecting to Listening, and select + `Show transcription in chat` before speaking. +- **1:55–2:40 — Describe the process.** Answer the first interview question in ordinary language. Once the answer is finalized, point out its visible text and Voice provenance without pausing the flow. -- **2:05–2:55 — State the decision and constraints.** Let the assistant ask a +- **2:40–3:30 — State the decision and constraints.** Let the assistant ask a narrower follow-up. Explain the purchasing trade-off and the three success criteria. -- **2:55–3:35 — Define one experiment.** Ask to compare the baseline policy +- **3:30–4:05 — Define one experiment.** Ask to compare the baseline policy against fragile supply over two years. -- **3:35–3:55 — Show recoverability.** Briefly identify mute, repeat question, +- **4:05–4:20 — Show recoverability.** Briefly identify mute, repeat question, read full response, and end controls. Do not demonstrate every control. -- **3:55–4:15 — Close on the UX value.** Emphasize progressive disclosure: +- **4:20–4:40 — Close on the UX value.** Emphasize progressive disclosure: ground the interview in the live net, speak naturally, inspect the finalized answer, then refine the structured model. @@ -100,10 +101,10 @@ ask a question that is relevant to what is already on the canvas.” **[Wait for Brunch to inspect the live net and ask its first relevant interview question]** -**Assistant, expected question:** “Walk me through the process you want to -model.” +**Assistant, expected question:** “How do Sonaflozin and Flowbind move through +Site 1000’s purchasing process before Sonic Flow production?” -**[1:00 — Select Voice]** +**[1:15 — Select Voice]** “Now that Brunch has inspected the model and asked the first interview question, I’ll switch to Voice. @@ -113,11 +114,18 @@ live audio, while Petrinaut keeps the finalized answers in the conversation rather than storing the audio. I can check the microphone, give consent, and start.” -**[Start Voice; let Connecting change to Listening]** +**[Give consent and start Voice; let Connecting change to Listening]** “The dock makes the current state clear. It connects, then tells me when it is listening.” +**[Select `Show transcription in chat`]** + +“Transcription starts hidden, so I’ll show it in the chat before I answer. Once +each answer is finalized, its text will remain visible in the conversation.” + +**[1:55 — Answer the first question in Voice]** + **Presenter, speaking to Voice:** “Site 1000 produces one finished product called Sonic Flow. It needs two raw @@ -134,6 +142,8 @@ without material, while excess stock costs money and may expire.” waveform marker shows that this answer came from Voice. I can inspect the exact text instead of relying on an invisible audio history.” +**[2:40 — State the decision and constraints]** + **Assistant, expected follow-up:** “What decision should the model help you make?” @@ -157,6 +167,8 @@ planned unit, and keep the expiry rate of each raw material below 5 percent.” decision and measurable constraints. I did not need to begin with formulas, token types, or a list of forty-five transitions.” +**[3:30 — Define one experiment]** + **Assistant, expected follow-up:** “What would you like to test first?” **Presenter, speaking to Voice:** @@ -166,13 +178,13 @@ under fragile supply, where supplier outages are more frequent. Show the effect on total policy cost, fill rate, holding cost, late-delivery penalties, and supplier switching.” -**[Point briefly to the dock controls]** +**[4:05 — Point briefly to the dock controls]** “The dock stays compact while the session is active. I can mute the microphone, show or hide transcription, repeat the current question, hear the full response, or end Voice without leaving the model.” -**[End Voice; leave finalized conversation visible]** +**[4:20 — End Voice; leave finalized conversation visible]** “The intended UX is progressive disclosure: ground the interview in the live net, speak naturally, preserve the finalized answer as inspectable text, and