feat(agent)!: preserve durable communication and independent runtimes

[yanekyuk]

Changed

• Rebase onto pinned origin/dev (eeab341624fe67b2eeb623abf8f8ed1afc20b5c6) while preserving the original feature history in backup refs. Keep dev's independent network approvals/invites, opportunity routes, Redis Streams, migration baseline, and web/Mac/Hermes improvements.
• Keep @indexnetwork/agent as the API's default hosted runtime, owned by PersonalAgentService. Keep packages/agentv2 and its external runner independent: no merged scheduler, third agent, runtime-selection framework, or checkpoint compatibility path.
• Reconstruct our agent's H2A/A2A context from durable principal records. Preserve private standing mandates, complete per-opportunity delegations, source provenance, principal-context freshness, exact question batches/retirement, and explicit creation/broadcast/resume/input/manual-review activations. A2A activity and local pauses do not wake our H2A.
• Preserve both TUI modes and explicit-query packages/discovery, without HyDE. Opening a hosted negotiation and its first private delegation commits atomically. Numbered negotiation sessions retain separate previous-session history and authoritative opportunity status.
• Preserve v2's own first-contact, briefing, outreach, independent-question, and stall-wake policies. It may reach counterparties before answers and briefs each new opportunity independently. Recover every signal's openings, owed nonzero turns, and missed principal-input batches; prevent startup adoption from dropping observed input/creation events.
• Share authoritative protocol actions and transactional executor ownership. Only the session's original responder may accept; agent agreement remains pending separate human approval. External messages, turns, and openings are fenced against executor handover.
• Adopt Redis Streams as transport, not runtime authority. Independent hosted readers observe retained events; SQL ownership and activation receipts fence effects. External named consumers replay pending pages. Transport acknowledgement never certifies successful reasoning, and retention does not provide indefinite/exactly-once execution.

Hosted availability and resume follow-up

• Restore an idle, sendable hosted runtime after a failed review without replaying that review. Transient startup failures retry with capped backoff instead of becoming permanently unavailable; explicit activations survive startup/replacement.
• Map explicit ACTIVE lifecycle transitions to version-fenced, deduplicated intent.resumed receipts. Stale/duplicate resumes do not wake again; resume cannot answer or retire pending questions or create new principal authority.
• Deliver web agent-status, lifecycle and question changes immediately over SSE. Reduce new-owner stream discovery from a 15-second wait to one second and remove a redundant startup record read.
• Log hosted failures with their cause. Local investigation identified a stale inherited tmux OpenRouter key overriding a valid worktree file key; only the local dev process was restarted without that inherited variable. No key values or infrastructure settings were changed.

Web H2A latency and question follow-up

• Fix refresh starvation: polls and SSE events now coalesce behind one read rather than invalidating every slower response. This reproduces and fixes the reported plain question bubbles with no answer controls and prolonged Connecting state.
• Release message/answer spinners after acceptance, not after a second GET. Preserve drafts on failed writes and require the exact whole hosted question batch.
• Read canonical pending questions in one SQL snapshot, without reconstructing profile/network/executor model context. Remove redundant input pre-reads while retaining transactional owner/executor/lifecycle checks and canonical batch validation. Effect/turn freshness fences remain unchanged.
• Show current hosted work as one plain-English, unboxed status line outside the scrolling transcript: Thinking…, Updating your brief…, Looking for matches…, and other tool-specific descriptions. No accordions, dropdowns, raw tool names, or retained completed-tool list. The subtle activity dot respects reduced motion. Stale/failed review notices remain separate; failed-review notices survive idle runtime replacement.
• Show the same live status for explicit resumes, including before the first tool and when there is no visible chat message or usable message anchor. Status clears when the review ends, without fabricating a reply or outcome. Tool observations are ephemeral: pause/runtime restart clears them; reopening a conversation does not replay old model work.
• Add the same explicit Wake action at authenticated POST /conversations/:id/wake with { intentId }. It commits a private activation receipt, not a message, answer or permission. Other owners, inactive intents and selected external executors cannot be woken through this hosted route. Existing external-agent policy remains independent.

TUI startup follow-up

• Fix the reproduced launch failure during a concurrent rm -rf dist protocol build. The root scenario command delegates to the package-local Bun launcher and resolves agent/discovery/protocol source entry points instead of rebuilding shared outputs. The existing env -u OPENROUTER_API_KEY bun --env-file=.env.development run agent:tui command is unchanged for users.
• Keep workspace dependencies external in the TUI library build and clear source aliases for declarations. The API-backed TUI still uses normal built-package exports and existing ownership fences; it must not compete with the running API for hosted seats.

Native Hermes and clients

• Port Seref's native think/speaker sessions to the retained agent through NegotiationSpeaker. Supply fresh prompts and exact tools; serialize session tools, interrupt on completion/cancellation/uncertain turns, and never retry an uncertain turn write.
• Replace obsolete checkpoint storage with private, atomically replaced FilePrincipalRecords: domain messages, briefs/delegations, retirements, activation and delivery receipts. Failed persistence cannot publish unsaved effects; lost publication responses replay the same output IDs rather than rerunning the authoring model.
• Require INDEX_EXECUTOR_ID on the Hermes/v2 device to identify its registered, selected external agent. Hermes does not take over an arbitrary external selection; v2 refuses to start without the binding.
• Reconcile web, Mac, CLI, Claude and Hermes consumers around { status, pending } and POST /conversations/:id/answers. Hosted submission accepts exactly the complete current batch. External executors retain their independent answer policy. Direct messages never implicitly answer questions. Preserve local drafts until accepted writes.
• Add packages/hermes-plugin/dashboard/index.js as source and explicit dashboard/desktop build commands; regenerate shipped bundles rather than editing generated code.

Breaking contracts and migration

• Remove PrincipalState/PrincipalStore checkpoint APIs and obsolete runtime storage. Hosts implement PrincipalRecords, session-aware negotiation reads, and guarded effect/turn/opening operations. No old checkpoint loader or compatibility layer remains.
• Apply 0003_add_principal_records_and_negotiation_sessions.sql after dev's 0000–0002 baseline. Preserve historical displayed questions, scoped evidence and advisory review notes; do not invent standing authority or retire later dev-authored questions using older checkpoint snapshots. Drop agent_sessions; add standing-brief pointers and negotiation-session identities using unprefixed tables. Do not concatenate the old feature migration journal.
• Unbriefed hosted intents require a permitted fresh review to become match-ready. Historical model conversations are not resumable authority.
• Release metadata/pins are aligned: protocol 60.0.0, API 0.131.0, agent 0.14.0, v2 0.3.4, client 0.6.0, TUI 0.12.0, web 0.81.0, CLI 0.28.0, Hermes 0.47.0, Claude 0.8.0; root and standalone CLI lockfiles are synchronized.

Verification

• 44 existing tests passed: agent 26, client 9, provider-free protocol 9. Agent/client suites reran after the workflow fixes.

• Passed affected agent/v2/client/discovery/protocol/API/TUI builds and typechecks, web production build, four-target CLI build, native Mac ad-hoc build, Hermes runtime/dashboard/desktop builds, Python compilation, and Hermes plugin doctor (23 tools, one hook).

• Passed protocol boundary checks, adapter naming, subtree/lockfile parity, and root lint (0 errors, 35 warnings). Commit hooks passed staged lint and their build gates. TUI/discovery runtime implementation remains preserved; the scenario launcher now avoids compiled-output races.

• Disposable real PostgreSQL/pgvector: fresh migration, dev-baseline fixture upgrade, repeat migration, and no-schema-drift generation.

• Real disposable PostgreSQL/Redis authority probes: exclusive hosted ownership, exact/concurrent batches, stale effects/turns, private records, executor handover, idempotent publication, one input pointer per committed external batch, atomic opening/delegation and cancellation rollback, responder-only acceptance, separate human approval, and successor-session history.

• Actual v2 host/runner and actual NegotiationAgent workflow probes verified their different briefing, question, pause/wake and resumption policies. Negative probes reproduced and then passed the reconnect/startup and stale-executor-opening fixes. These use real SQL/API controller methods and HTTP client serialization with fixture authentication/model responses and fixed embeddings, not a full authenticated deployment.

• Real isolated Redis replay checks included 205 pending entries, independent consumers, stable identities and concrete live cursors. Native Hermes AIAgent/registry/session-store probes used loopback model replies to verify fresh context, tool schemas and hard termination. File-record probes covered restart, freshness, failed persistence and lost-response replay.

• Follow-up verification: real PersonalAgentService with disposable SQL/Redis reproduces the original failure and passes idle recovery, explicit resume, stale/duplicate/idempotent rejection, restart replay, question-retirement refusal, four transient startup failures, and transient receipt-write recovery. Existing agent tests (26), hosted/v2 workflow probes, affected typechecks/builds, staged lint and lockfile/subtree parity passed.

• Actual Chromium checks cover the built production SPA with loopback API/auth fixtures. Status/lifecycle/executor changes became visible in 25–26 ms with five-second polling disabled. Question batching/draft retention, direct-message separation, mobile panels and negotiation history also passed. This is not live-backend end-to-end verification.

• Web H2A follow-up: Chromium reproduced the bug with 6.2-second GETs before the fix, then verified question controls and independent send completion after it. Real EventSource (polling disabled) verified plain-English live activity, idle completion, review notices, Wake and external-executor visibility. Existing batch/correction/draft/mobile/history checks passed.

• Real disposable SQL/Redis plus actual controller/service methods verified acceptance before model completion, review activity frames, complete/partial answer batches, private Wake receipts, cross-owner/inactive/external rejection, and visible provider-failure recovery without model replay. Hosted/v2 authority probes and the 26 existing agent tests passed again.

• Read-only feature-clone measurements: pending-state reconstruction dropped from 9 queries/~2.0 s to 1/~0.28 s; warm GET-controller time dropped from ~3.5 s to ~1.8 s. These exclude AuthGuard and do not measure live POST/model latency. Root lint (0 errors/35 warnings), API/agent/TUI/Hermes checks, web production build, parity and commit build gates passed.

• Plain-English status follow-up: the production SPA with real EventSource and polling disabled passed all four tool-description mappings, pre-tool/between-tool Thinking, absent/old anchors, long scrolled history, desktop/mobile layout, reduced motion, and completion without an invented chat reply. Component batch/draft checks, scoped lint, web build, and the unchanged typecheck-baseline comparison passed.

• Actual service/controller methods with disposable SQL/Redis verified fresh resume activity before a first tool and with no visible message, real status frames, duplicate-receipt rejection, pause clearing observations, and a later explicit resume without replay. Read-only feature-clone receipts confirmed earlier resumes were accepted. No additional real intents were resumed for these checks.

• TUI follow-up: the actual root launcher passed while protocol/dist/index.js was absent during a concurrent rebuild. Both root/package help commands and the interactive scenario chooser passed with compiled workspace imports forbidden and zero network/model calls; Esc exited without selecting a scenario. TUI checks/build, API typecheck, protocol architecture, lockfile checks and mandatory commit build gates passed. The API-backed mode was built/typechecked, not launched as a second hosted seat.

Detailed evidence and scope are in REBASE_TODO.md; temporary probe scripts/logs are preserved outside the source tree. No repository tests were added.

Remaining limits

• Not run for this rebased integration: live-provider model evaluation, interactive TUI agent conversations, and native device checks. Provider-key authentication was verified locally, but builds and fixture replies do not verify model judgment or every UI/runtime path. The Mac build was ad-hoc signed, not notarized.
• A separate pre-existing web defect remains: Edit → Refine calls the removed /intents/:id/refine route. The mismatch is already present in pinned dev and is outside this availability/resume fix.
• Whole-web tsc has 34 pre-existing diagnostics. A compiler-host comparison with the exact pre-fix versions of both changed web files reports an identical diagnostic set: no new errors, not a clean whole-web typecheck. This unrelated baseline is not repaired in the H2A fix.
• Natural-language authority remains model-guided. V2 intentionally retains wake-driven brief updates, not our hosted principal-context invalidation semantics. Hermes local-file/REST checks are not transactionally equivalent to hosted SQL checks.
• No merge, deployment, or shared/production migration. Keep this PR draft pending review.

Related design discussion: #1612. The implemented preservation plan supersedes its original consolidation proposal.

[yanekyuk]

Live interactive validation: responder-only acceptance — passed

Ran the actual agent:tui entry point in an isolated tmux session with live OpenRouter model/embedding calls and a temporary two-user fictional scenario. Interacted through the scenario chooser and H2A text inputs; no model replies or protocol decisions were mocked.

• Counteroffer / acceptance: Alice (initiator) proposed 60 minutes; Bob (responder) countered with 30. Alice paused until principal permission. Even when the H2A message explicitly asked Alice to “accept his counteroffer,” her agent explained the initiator restriction and submitted counter, not accept. Bob then submitted accept. The TUI showed agreed / pending human review.
• Initiator withdrawal: opened a separate new session with a firm 60-minute requirement and explicit permission to withdraw on a 30-minute counteroffer. Actual turns: Alice propose → Bob counter → Alice decline. The TUI showed declined / rejected, while the previous agreement remained agreed / pending.
• Verified the exact seven recorded turns and both session statuses against the application's exported transcript and terminal captures. Ctrl+C exported the transcript and exited 0.

No code changes were needed. The temporary fixture and tmux session were cleaned up; the worktree is clean. This adds live agent/TUI coverage to the earlier deterministic checks, but does not exercise authenticated backend HTTP/Postgres/Redis integration or inject a forged forbidden action.