openpi-dev · tt-a1i · Sep 8, 2026 · Sep 8, 2026
diff --git a/docs/research/CURSOR_RELIABILITY_BASELINE_2026-09-08.md b/docs/research/CURSOR_RELIABILITY_BASELINE_2026-09-08.md
@@ -0,0 +1,122 @@
+---
+status: draft
+created: 2026-09-08
+last-verified: 2026-09-08
+applies-to: isolated v0.8.1 worktree a3edee28 plus local OPENPI_CURSOR_WIRE_LOG instrumentation; not the user's daily runtime
+related-issues: "#234"
+related-prs: "#484, #485"
+supersedes: none
+---
+
+# OpenPI Cursor reliability experiment baseline
+
+Exploratory research setup and first live samples. Not a formal Benchmark.
+
+## Question
+
+Can parent-agent and subagent tasks on Cursor Grok configs be systematically accepted against machine-checkable acceptors on released OpenPI v0.8.1, and which failure classes are OpenPI-fixable?
+
+## Observed facts
+
+### User checkout (untouched)
+
+Commands run in `/Users/tushaokun/work/my-pi-setup` without writing to it:
+
+- Branch: `codex/knowledge-evidence-contract`
+- HEAD: `a9b40f0044ee59c360a6077c1c0bdbdbd30da10b`
+- Dirty: yes (docs/research, architecture visuals, `.hive/`, repro scripts). Left intact.
+- Local `main`: `2a69d3f32994da4123f1312b7fa84ef3d6119be1` (stale vs origin)
+- `origin/main`: `0d17f4577fe31315fe6c95370d251bdb4e2413cf` (contains v0.8.1)
+
+### Release identity
+
+- Annotated tag `v0.8.1` → `a3edee28fec813582db6b3b1341754cef4597f81` (`chore(release): bump version to 0.8.1 (#485)`, 2026-09-08)
+- Package version at that commit: `0.8.1`
+- Installed Pi CLI: `0.85.1` (`/Users/tushaokun/.bun/bin/pi`)
+
+### Unique daily OpenPI source (do not use for these experiments)
+
+`pi list` (user config, no `PI_CODING_AGENT_DIR`):
+
+```
+User packages:
+  npm:pi-web-access
+    /Users/tushaokun/.pi/agent/npm/node_modules/pi-web-access
+  ../../work/openpi-main-runtime
+    /Users/tushaokun/work/openpi-main-runtime
+```
+
+One OpenPI source: `/Users/tushaokun/work/openpi-main-runtime` at `ed9dbc1018f890fd54375f5371990ddfee8af5df`, package `0.7.0`, branch `main` behind origin by 1. Not v0.8.1. User `settings.json` packages were unchanged after isolated install.
+
+### Isolated experiment assets
+
+| Asset | Path |
+| --- | --- |
+| Worktree | `/tmp/openpi-reliability-20260908/src` on `research/openpi-cursor-reliability-20260908` @ `a3edee28` |
+| Isolated Pi agent dir | `/tmp/openpi-reliability-20260908/pi-agent` via `PI_CODING_AGENT_DIR` |
+| Synthetic git repo | `/tmp/openpi-reliability-20260908/synthetic` (HEAD `509b2472d17f50411916ee77c44ec09301b129e4`) |
+| Harness | `/tmp/openpi-reliability-20260908/harness` |
+| Results | `/tmp/openpi-reliability-20260908/results` |
+| Continuation | `/tmp/openpi-reliability-20260908/CONTINUATION.md` |
+
+Isolation probe: `PI_CODING_AGENT_DIR` empty dir listed `No packages installed`; user packages and 939 user Sessions unchanged.
+
+Isolated `pi install` of the worktree then listed only:
+
+```
+User packages:
+  ../src
+    /tmp/openpi-reliability-20260908/src
+```
+
+Credentials: isolated `auth.json` is a copy of the user file (providers present: cursor, kimi-code, openai-codex, xai). User `auth.json` / `models-store.json` / `settings.json` mtimes were not changed by isolated `pi update --models` or catalog seed.
+
+### Cursor Grok config availability
+
+Targets: `cursor-grok-4.6-high`, `cursor-grok-4.6-high-fast`, `cursor-grok-4.6-medium-fast`. No substitutions.
+
+| Probe | Result |
+| --- | --- |
+| User `pi --list-models cursor-grok` | All three IDs present (account catalog, 217 Cursor models, `checkedAt` 1788861139372) |
+| Isolated `pi update --models` | Refreshed built-in `openai-codex` and `xai` only. No `cursor` key written. |
+| Isolated `pi --list-models cursor` before seed | Only static `cursor/default` from OpenPI |
+| Isolated `fetchCursorUsableModels` | Failed: `discovery_transport_or_protocol_failure` in 1141 ms (`results/live-discovery.json`) |
+| Isolated `pi --list-models cursor-grok` after seeding the three IDs + `default` from the user catalog | All three IDs listed |
+| `pi auth check --provider cursor` | `provider_not_found` in both user and isolated CLIs (Cursor is an OpenPI-registered provider, not a Pi built-in auth provider) |
+
+Availability for selection in the isolated CLI is therefore: **listed after catalog seed**, **not confirmed by live GetUsableModels**. First live AgentService run (below) did stream on `cursor-grok-4.6-high-fast`, which is stronger than list-models alone for that one ID.
+
+### First live parent sample (cell A)
+
+- Run: `results/cells/A-cursor-grok-4.6-high-fast-01-1788881803136`
+- Model: `cursor-grok-4.6-high-fast`, thinking `high`, tools `read,bash,rg,fd`
+- Duration: 180059 ms, harness SIGTERM, exit 143
+- Ledger classification: `timeout`
+- Acceptor `A-read-search-bash` v `2026-09-08.1`: `ACCEPT_A.txt` missing; nonce file and forbidden file unchanged
+- Native wire log: none (no `OPENPI_CURSOR_WIRE_LOG` events)
+- Discriminative evidence: 24 turns, 46 successful Pi tool executions (`read` 23, `rg` 23, 0 errors). The model repeatedly stated it had used Cursor-native / wrong wrappers and must switch to "openpi MCP tools", then called `read`+`rg` again. It never called `bash` and never wrote `ACCEPT_A.txt`.
+- Refined class: **model strategy failure** that the harness cut off as **timeout**. Not an agent connection timeout, not a remote disconnect, not a native-rejection hang.
+- Likely related OpenPI text: `CURSOR_PI_TOOLS_SYSTEM_PROMPT` tells the model to use "openpi MCP tools" and not Cursor-native tools. Advertised names remain `read` / `rg`. Whether that wording induces the loop is **under investigation**.
+
+### Limits
+
+- Matrix not complete. One parent cell, one model, one attempt.
+- Live usable-model discovery failed; catalog seed is not a substitute for GetUsableModels.
+- User daily runtime remains OpenPI 0.7.0 at `ed9dbc10`. These runs do not accept that runtime.
+- Isolated worktree has uncommitted env-gated `OPENPI_CURSOR_WIRE_LOG` instrumentation in `extensions/ai-providers/cursor/provider.ts`. Behavior is unchanged when the env var is unset.
+- Formal Benchmark identities are not claimed.
+
+## Inferences
+
+The v0.8.1 native-rejection path was not exercised in this A sample. The blocking live failure seen so far is a tool-identity loop after successful Pi MCP calls.
+
+## Unknowns
+
+- Whether high / medium-fast show the same loop
+- Whether a prompt naming the actual tool IDs stops the loop
+- Why isolated GetUsableModels failed while AgentService/Run streamed
+- Cancel / disconnect / timeout / cleanup deterministic suite not yet re-run in this worktree this session
+
+## Next
+
+See `/tmp/openpi-reliability-20260908/CONTINUATION.md`.
diff --git a/docs/research/CURSOR_RELIABILITY_E512_VERSION_SPLIT_2026-09-09.md b/docs/research/CURSOR_RELIABILITY_E512_VERSION_SPLIT_2026-09-09.md
@@ -0,0 +1,18 @@
+---
+status: draft
+created: 2026-09-09
+last-verified: 2026-09-09
+applies-to: isolated E high-fast receipts; exploratory, not a Benchmark
+related-issues: "#512"
+related-prs: "#515"
+---
+
+Canonical machine+human index lives outside git:
+
+- `/tmp/openpi-reliability-20260908/results/e-high-fast-512-version-split.md`
+- `/tmp/openpi-reliability-20260908/results/e-high-fast-512-version-split.json`
+- `/tmp/openpi-reliability-20260908/results/source-overlay.json`
+
+**E-01..10** = pre-fix stock spawn / `a3edee28` behavior (7/10). E-06..10 are in this window.
+
+**E-11..20** = post-fix spawn text from commit `7950570` (10/10). Receipt `source` fields on those rows are still the hardcoded stock SHA; do not treat that field as product HEAD.
diff --git a/docs/research/CURSOR_RELIABILITY_PROGRESS_2026-09-08.md b/docs/research/CURSOR_RELIABILITY_PROGRESS_2026-09-08.md
@@ -0,0 +1,86 @@
+---
+status: draft
+created: 2026-09-08
+last-verified: 2026-09-09
+applies-to: isolated v0.8.1 worktree a3edee28 plus print-host wiring and unused-unless-set OPENPI_CURSOR_WIRE_LOG; exploratory, not a Benchmark
+related-issues: "#234, #512, #513, #514, #516"
+related-prs: "#484, #515"
+supersedes: none
+---
+
+# OpenPI Cursor reliability — session progress
+
+Companion to [`CURSOR_RELIABILITY_BASELINE_2026-09-08.md`](CURSOR_RELIABILITY_BASELINE_2026-09-08.md). Ledger: `/tmp/openpi-reliability-20260908/results/ledger.jsonl`. Not a formal Benchmark.
+
+## Matrix (first 5 attempts per cell; all attempts kept)
+
+| Config | A | B | C | D | E | F |
+| --- | --- | --- | --- | --- | --- | --- |
+| cursor-grok-4.6-high-fast | 5 (3 correct, 2 identity-loop timeout) | 5/5 | 5/5 native≥1 | 5 (4 correct, 1 error-retry timeout) | first 5: 2 correct, 3 print-settle. E-06..10 still stock spawn (5/5 correct). E-11..20 print-host after-fix (10/10) | 5/5 |
+| cursor-grok-4.6-high | 5/5 | 5/5 | first 5: 4 native-occurring (3 correct, 1 wrong); C-03 no-native. Extra C-06 native+correct → 5 counting | 5 (4 correct, 1 timeout) | 5/5 | 5/5 (+5 extra correct) |
+| cursor-grok-4.6-medium-fast | 5 (4 correct, 1 hang-after-success timeout; acceptor still green) | 5 (4 correct, 1 hang-after-success timeout; acceptor still green) | first 5: 4 native-occurring (C-01 timeout, no native). Extra C-06 native+correct → 5 counting | 5 (2 correct, 3 timeout) | 5/5 | 5/5 |
+
+Class C counts only when a native request occurred (`native_rejection` on the wire). high-fast C is 5/5. high and medium-fast each now have ≥5 native-occurring runs after extras (old no-native rows kept).
+
+## Long tasks (≥10 min wall + independent external acceptor)
+
+| Run | Duration | Acceptor | Class |
+| --- | --- | --- | --- |
+| LP-high-fast-01 | 789s | pass (4 shard path hashes) | correct completion |
+| LH-medium-fast-01 | 633s | fail (count/hash mismatch) | wrong result |
+| LH-high-01 | 819s | fail | wrong result |
+| LD-medium-fast-01 | 791s | fail (15–16 line / hash drift) | wrong result |
+| LD-high-fast-01 | 1165s | fail | wrong result |
+| LD-high-01 | 1710s | fail (15/16 lines, hash drift) | wrong result |
+
+Shorter audits (L 210s, other LPs 217–487s) have acceptors but do **not** count toward the six 10-minute tasks.
+
+Content-hash long tasks often mismatch because models include an extra `.ts` file or hash a different canonicalization. That is model-strategy / wrong-result, not infrastructure. One path-list pipeline passed.
+
+## Criterion 3 — cancel / disconnect / timeout / cleanup
+
+- Unit (re-run 2026-09-09, logs captured): `results/fault/pi-backend-lifecycle.unit.log` 20 pass / exit 0; `results/fault/cursor-provider.unit.log` 30 pass / exit 0.
+- Live cancel `cancel-live-1788887430222`: `sawTool=true`, SIGTERM after first `tool_execution_start`, leftover `[]`, user Sessions 939, class `cancel`.
+- Live disconnect `disconnect-live-1788887477808`: isolated CONNECT proxy dropped after first tool; next stream `stopReason: error` / `Client network socket disconnected before secure TLS connection was established`; then `agent_settled`; leftover `[]`; Sessions 939. Class `infrastructure failure`.
+- After harness SIGTERM loops: no leftover `run-cell` processes. Isolation leak from an earlier un-prefixed lifecycle test was removed; Sessions restored to 939.
+
+## Failure classes (all attempts kept)
+
+1. **Tool-identity loop** (A high-fast 01, 04) — [#513](https://github.com/openpi-dev/openpi/issues/513). Many successful Pi `read`/`rg`; model claims native / `mcp_openpi_*` names; never writes ACCEPT_A. Under investigation. Not connection timeout.
+2. **Hang after acceptor success** (A-04, B-02 medium-fast) — [#516](https://github.com/openpi-dev/openpi/issues/516). Work written, session does not exit; harness timeout. Under investigation.
+3. **Error-recovery loop** (D timeouts) — repeated failed missing-path reads; never ACCEPT_D. Model strategy.
+4. **Print-mode parent settle** (E high-fast 01–03) — [#512](https://github.com/openpi-dev/openpi/issues/512). Spawn then `stop` without wait; `--print` emits `agent_settled`. Wiring on research branch / [PR #515](https://github.com/openpi-dev/openpi/pull/515) (not merged). Version-split index: `/tmp/openpi-reliability-20260908/results/e-high-fast-512-version-split.md`.
+5. **Native reject without task recovery** (C high 02) — rejection recorded, zero Pi tools, wrong result.
+6. **Insufficient evidence** (C high 03; first live cancel) — no tools / no native frames.
+7. **Long-task hash canonicalization** — systematic extra-file or digest drift on content-hash shards.
+8. **Explorer inherits write** — [#514](https://github.com/openpi-dev/openpi/issues/514). Advertised `write`/`bash` on a “read-only” role. No `CHILD_ESCAPED.txt` in E samples.
+
+## Print-host fix (#512)
+
+Before (stock v0.8.1 spawn text), same E task+acceptor, `cursor-grok-4.6-high-fast`, **E-01..10** (not E-06..20): **7/10** correct. E-06..10 are still pre-fix.
+
+After (print-host spawn text + bounded `followUp` barrier), **E-11..20**: **10/10** correct. E-11 spawn result: `This host cannot re-invoke you after you end the turn. You MUST call subagent_wait...`
+
+Receipt `source` fields remain `a3edee28` on those 20 rows (harness hardcoded). Overlay: `results/source-overlay.json`. New C extras record HEAD `7950570`.
+
+Isolated `bun run check` exit 0 (`results/isolated-bun-check.log`) and `bun run test` exit 0, 1470 pass (`results/isolated-bun-test.log`).
+
+No tools disabled, permissions unchanged, no timeout lengthening, no swallowed errors.
+
+## Issues
+
+- [#512](https://github.com/openpi-dev/openpi/issues/512) print-mode settle — evidenced; wiring on [PR #515](https://github.com/openpi-dev/openpi/pull/515), not merged.
+- [#513](https://github.com/openpi-dev/openpi/issues/513) Cursor prompt vs advertised names — under investigation; no 10/10 A-loop before-after yet (only 2 failing / 5 on high-fast A).
+- [#514](https://github.com/openpi-dev/openpi/issues/514) explorer inherit write — under investigation; no proven write escape.
+- [#516](https://github.com/openpi-dev/openpi/issues/516) hang-after-success on print after green acceptor — under investigation.
+
+## Sample limits
+
+- Baseline cells n=5. E high-fast n=20 for the #512 before/after only.
+- Cursor usage fields are typically zeros / unknown.
+- GetUsableModels probe without `ALL_PROXY` failed; live `pi --print` with the user proxy inherited streamed all three Grok IDs.
+- Concurrent parents: 3 long heavies in parallel produced hash errors, not connection storms.
+
+## Isolation
+
+Always `PI_CODING_AGENT_DIR=/tmp/openpi-reliability-20260908/pi-agent`. Unique isolated OpenPI: `/tmp/openpi-reliability-20260908/src`. User packages (`pi-web-access` + `openpi-main-runtime`) and 939 Sessions unchanged.
diff --git a/docs/research/CURSOR_RELIABILITY_SUMMARY_2026-09-08.md b/docs/research/CURSOR_RELIABILITY_SUMMARY_2026-09-08.md
@@ -0,0 +1,44 @@
+---
+status: draft
+created: 2026-09-08
+last-verified: 2026-09-09
+applies-to: isolated OpenPI v0.8.1 experiments; exploratory, not a Benchmark
+related-issues: "#512, #513, #514, #516"
+related-prs: "#484, #515"
+supersedes: none
+---
+
+# OpenPI Cursor reliability — summary
+
+Exploratory. Not a formal Benchmark. Full matrix and classifications: [`CURSOR_RELIABILITY_PROGRESS_2026-09-08.md`](CURSOR_RELIABILITY_PROGRESS_2026-09-08.md).
+
+## What was verified
+
+- **Runtime:** Pi 0.85.1; experiment source v0.8.1 `a3edee28`; unique isolated package `/tmp/openpi-reliability-20260908/src`. Daily OpenPI remains `openpi-main-runtime` @ `ed9dbc10` (0.7.0) — unused.
+- **Configs:** `cursor-grok-4.6-high`, `cursor-grok-4.6-high-fast`, `cursor-grok-4.6-medium-fast` all listed and all streamed live. No silent substitution.
+- **18 baseline cells × 5** on the three configs (A–F). Class C native-occurring counts: high-fast **5/5**; high **4/5 then +C-06 → 5**; medium-fast **4/5 then +C-06 → 5**. Old no-native rows kept.
+- **Six 10+ minute tasks** with external acceptors (LP-high-fast-01 path-hash pass at 789s; five content-hash runs 633–1710s with evaluated wrong-result acceptors).
+- **Cancel / disconnect / timeout / cleanup:** unit logs recaptured (`results/fault/*.unit.log`, 20/20 and 30/30, exit 0); live cancel/disconnect unchanged; leftover processes empty; user Sessions 939.
+- **#512:** E-01..10 stock spawn 7/10; E-11..20 print-host 10/10. Version-split index in `results/e-high-fast-512-version-split.md`. Isolated bun logs: `results/isolated-bun-check.log` / `isolated-bun-test.log` (both exit 0). PR #515 not merged.
+
+## What remains unreliable
+
+- high-fast class A identity loop (~2/5). [#513](https://github.com/openpi-dev/openpi/issues/513). No 10/10 before-after yet.
+- medium-fast hang-after-success (acceptor green, process does not exit). [#516](https://github.com/openpi-dev/openpi/issues/516).
+- D error-retry loops (especially medium-fast 3/5 timeout).
+- C on high: 3 correct + 1 wrong among 5 native-occurring (plus earlier no-native C-03).
+- Long content-hash acceptors (model canonicalization).
+- Explorer/reviewer inherit write/bash. [#514](https://github.com/openpi-dev/openpi/issues/514).
+
+## Before / after (#512 only)
+
+| Source | Attempts | n | Correct |
+| --- | --- | --- | --- |
+| v0.8.1 stock spawn (E-01..10; E-06..10 still this window) | E / high-fast | 10 | 7 |
+| print-host `7950570` spawn text (E-11..20) | E / high-fast | 10 | 10 |
+
+Other cells were not re-run after the fix except in-flight longs.
+
+## Sample limits
+
+n=5 per baseline cell. Usage unknown. Live discovery without proxy failed. Do not treat this file as a Benchmark.
diff --git a/docs/research/README.md b/docs/research/README.md
@@ -10,6 +10,14 @@ Research records preserve sourced investigation and distinguish observations, in
 
 - [`CURSOR_NATIVE_RECOVERY_2026-09-08.md`](CURSOR_NATIVE_RECOVERY_2026-09-08.md) — bounded in-band native execution rejection, Pi-owned tools, cancellation identity, and actual child acceptance ([#234](https://github.com/openpi-dev/openpi/issues/234)).
 
+- [`CURSOR_RELIABILITY_BASELINE_2026-09-08.md`](CURSOR_RELIABILITY_BASELINE_2026-09-08.md) — isolated v0.8.1 experiment baseline, Cursor Grok availability probes, and first parent-cell evidence (exploratory; not a Benchmark).
+
+- [`CURSOR_RELIABILITY_PROGRESS_2026-09-08.md`](CURSOR_RELIABILITY_PROGRESS_2026-09-08.md) — live matrix progress, failure classification, #512 before/after, and remaining cells (exploratory; not a Benchmark).
+
+- [`CURSOR_RELIABILITY_SUMMARY_2026-09-08.md`](CURSOR_RELIABILITY_SUMMARY_2026-09-08.md) — configs/tasks verified, remaining unreliability, before/after deltas, sample limits (exploratory; not a Benchmark).
+
+- [`CURSOR_RELIABILITY_E512_VERSION_SPLIT_2026-09-09.md`](CURSOR_RELIABILITY_E512_VERSION_SPLIT_2026-09-09.md) — pointer to the E-01..10 vs E-11..20 overlay (exploratory; not a Benchmark).
+
 - [`WEB_STARTUP_2026-09-07.md`](WEB_STARTUP_2026-09-07.md) — terminal startup feedback, browser-launch waiting, and exploratory timing limits ([#450](https://github.com/openpi-dev/openpi/issues/450)).
 
 - [`WEB_STREAMING_MARKDOWN_2026-09-07.md`](WEB_STREAMING_MARKDOWN_2026-09-07.md) — unchanged historical Markdown parsing during streaming, bounded React component reuse and deterministic regression evidence ([#434](https://github.com/openpi-dev/openpi/issues/434)).