Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,10 @@ jobs:
src/wiki/operations/__tests__/identity-precision.test.ts
src/wiki/operations/__tests__/integrity.test.ts
src/wiki/index/__tests__/maintenance.test.ts
src/graph/__tests__/candidate-process.test.ts
src/graph/__tests__/fingerprint-store-transactions.test.ts
src/graph/__tests__/store-statements.test.ts
src/graph/__tests__/engine-continuity-resources.test.ts
- run: npm run build
- name: Smoke test the packed install
run: npm run test:hub:package
Expand Down
37 changes: 34 additions & 3 deletions .mex/ROUTER.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ edges:
condition: when starting a task — check the pattern index for a matching pattern file
- target: patterns/release-readme-visuals.md
condition: when refreshing the release README, badges, community links, or architecture illustrations
last_updated: 2026-09-08
last_updated: 2026-09-09
---

# Session Bootstrap
Expand Down Expand Up @@ -116,8 +116,11 @@ Then read this file fully before doing anything else in this session.
Ordinary Wiki creation/synthesis retain explicit provenance or capture the
operation actor/time/session; completed plain creates replay without duplicate
records. Legacy Team/Spec recovery bytes stay unchanged. This is local
implementation on `codex/0.8.1`; Settings heap calibration on the pinned runner remains an
explicit release gate alongside the existing release requirements.
implementation on `codex/0.8.1`. The three missing Settings heap limits are now
calibrated from retained pinned Linux run `34286120355`; a clean enforcing
run after correction remains a release gate. Evidence is in
`docs/design/settings-heap-calibration.json`. The separately accepted Graph
isolation timing calibration below changes only its five owned time limits.
- Phase 5 narrows shared artifact I/O to exact bytes by default, with explicit
checkout-neutral reads for canonical Team records. Wiki/local revisions remain
exact through Inbox preview and recovery; legacy Timeline IDs remain stable
Expand All @@ -132,6 +135,23 @@ Then read this file fully before doing anything else in this session.
the isolated Windows changes. Runner verification is attached to the PR.
- Explicit graph status, refresh, and isolated rebuild/recovery commands preserve
the last trustworthy index behind one cross-process maintenance lease.
- Graph performance work is implemented separately on
`codex/0.8.1-graph-performance`: outer-owned fingerprint publication, fixed
statement reuse, smaller continuity/reference staging, and disposable Hub
candidate construction. The parent retains validation and publication;
ordinary CLI construction remains in process. Implementation, verification,
and actual Hub measurements are in
`docs/design/code-graph-performance-implementation.md`. This targets the
release branch and is not a published release.
- Corrected graph performance PR #180 run `34288560611` passed Node 22/24,
browser, and Windows/macOS portability checks. Two independent pinned runners
confirmed five material Graph timing failures. The product decision accepts
disposable-worker startup latency for a responsive Hub and compiler-memory
release after jobs. Only those five time limits are recalibrated from retained
pinned evidence in `docs/design/graph-maintenance-timing-calibration.json`;
all memory and other limits stay unchanged. A clean enforcing run on the new
calibrated head remains required. Local same-code memory/latency evidence is
retained in `docs/design/graph-isolation-diagnostic.json`.
- Targeted graph get/query/impact consumers use one provenance-bound immutable
snapshot and discard output if graph or exact source identity changes.
- The graph half of Checkpoint 2 is working in the Project Hub: grouped symbol
Expand Down Expand Up @@ -228,6 +248,17 @@ Then read this file fully before doing anything else in this session.
- Public package-root exports for the provisional team contracts.

**Known Issues:**
- Graph construction still rebuilds the full eligible corpus after source
changes and has no peak-memory quota. The branch's actual Hub probe peaked at
about 1,963 MiB combined RSS and retained multi-second pauses around initial
checks and validation/publication despite responsive compiler-phase polling.
Fatal parent exit may leave owned temporary artifacts. The historical
`docs/design/code-graph-resource-investigation.md` explains the bottleneck and
retained-memory experiments; neither it nor the implementation rules out all
native, slow, or repository-specific leaks. Process isolation can increase
aggregate peak memory even while reducing memory left in the surviving Hub.
Corrected PR #180 platform and browser suites passed; runtime release
enforcement remains pending the accepted five-leaf timing calibration.
- Graph schema v4 is operational in this checkout, with two partially parsed
source files and no failed files. Treat partial graph evidence as degraded and
narrow or fall back to source discovery when needed.
Expand Down
20 changes: 17 additions & 3 deletions .mex/patterns/fresh-graph-hub-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ edges:
condition: "when changing Hub routes, sessions, safe projections, SSE, or jobs"
- target: "context/architecture.md"
condition: "when reviewing the current Graph-to-Hub architecture and boundaries"
last_updated: 2026-09-06
last_updated: 2026-09-09
mex:
id: mx_01M1M0CJKZF3ABC1PQREMA2HYR
type: pattern
status: promoted
revision: 4
revision: 5
title: fresh-graph-hub-integration
grounds_to:
- node: function:9099fdd7e5562f7507cc7e80a6d67f1e
Expand Down Expand Up @@ -57,7 +57,9 @@ or make Wiki availability appear real.
1. Bind one package-private repository adapter through
[`createRepositoryGraphPort()`](mex://function:9099fdd7e5562f7507cc7e80a6d67f1e).
Implement the frozen `GraphPort` by calling Lane A modules directly; do not add a package-
root export, raw SQLite callback, command escape hatch, or subprocess.
root export, raw SQLite callback, or public CLI subprocess. Production Hub
maintenance selects Lane A's private candidate process; read paths retain
their existing immutable-session contract.
2. Route every graph-derived response through the complete freshness handshake:
inspect a stable `fresh` graph, adopt one inode-bound immutable SQLite
session, read graph facts and hash-matched contained source, build the whole
Expand Down Expand Up @@ -86,6 +88,10 @@ or make Wiki availability appear real.
counts, retain the Hub generation/lease checks, and let Lane A's cross-process
maintenance lock arbitrate Hub and CLI writers. Rebuild requires the browser
confirmation step; neither operation runs during an ordinary read.
The parent owns the lease, candidate validation, publication and rollback.
The private child only constructs the candidate. Wait for process closure
before cleanup, terminate disconnected/cancelled children, and use a parent
lifeline independent of the busy compiler thread. Keep the public API fixed.
9. Derive Wiki availability independently from the registered adapter and its
current health. Keep unavailable states honest, and never fill Graph or Wiki
gaps with production fixtures.
Expand All @@ -107,6 +113,14 @@ or make Wiki availability appear real.
repair control.
- Job progress messages can contain paths or source details. Persist phase and
numeric counts only; discard the message.
- Graph counts describe parsed files. Keep persisted progress monotonic, label
those counts explicitly, and show a numeric bar only during parsing. The
final validation/publication phases do not inherit a 100% completion claim.
- Process isolation releases the child's working set when it exits; it is not
a peak RAM quota. Parent status inspections, copies and final publication
still include synchronous work. Measure actual HTTP/cancel latency as well
as aggregate resources. Fatal parent death stops the writer via its lifeline
but may leave private temporary artifacts; do not delete by a guessed prefix.
- Successful graph maintenance invalidates cached Search, Code, Health, Jobs,
Home, Overview, and capability queries. It does not authorize automatic
maintenance later.
Expand Down
53 changes: 48 additions & 5 deletions .mex/patterns/release-performance-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ edges:
condition: "when the regression involves Hub routes, browser sessions, or jobs"
- target: "patterns/safe-graph-snapshot-evolution.md"
condition: "when changing Graph maintenance or corpus inspection"
last_updated: 2026-09-03
last_updated: 2026-09-09
mex:
id: mx_01M1M0CJNG4SW0WCJF3NB547HE
type: pattern
status: promoted
revision: 3
revision: 6
title: release-performance-gate
grounds_to:
- node: function:5f86a557c717597b411a71a82c000ded
Expand Down Expand Up @@ -87,9 +87,29 @@ calibration environment.
pagination.
- Cross-tab job discovery must be event-driven. Do not restore continuous
polling to repair cache invalidation.
- Corpus byte caps prevent runaway allocation, but maintenance should also
release source bodies and parser state as each file or bounded compiler batch
completes.
- Corpus byte caps bound admitted input, not process RAM. Compiler dependencies,
AST/checker state, graph materialization and native storage can amplify that
input; measure peak working set separately from post-cleanup retained memory.
A low heap cap can abort extraction rather than make it memory-efficient.
- Synthetic fixture size is not repository scale. The current 48-file largest
fixture missed a real 708-file fingerprint write dominated by SQLite nested
savepoint bookkeeping. Separate fingerprint computation from persistence,
inspect native stacks, and verify both output equality and failure rollback.
See the [2026-09-09 investigation](../../docs/design/code-graph-resource-investigation.md).
- A maintenance child must be included in aggregate CPU/RSS measurements.
Measure real HTTP/cancel responsiveness and parent-owned temporary cleanup;
fatal heap exhaustion bypasses JavaScript `finally` blocks. Process isolation
alone is not a reduction in total work or peak memory.
The release sampler now sums Hub and observed descendants. RSS can count
shared pages twice; sampled CPU can miss short-lived children and final exit
work. The separate graph characterization exercises overlapping projects,
installed declarations, inferred JavaScript and actual executable edits;
it does not recalibrate the frozen release gate.
- Observe maintenance completion through the same bounded event subscription as
the Hub UI. Aggressive status polling opens extra SQLite readers once process
isolation makes the Hub responsive, adding observer work to the operation.
Keep POST-to-terminal elapsed time and child CPU/RSS included, retain the
absolute deadline, and record the observation method in report provenance.
- Back-to-back confirmation processes on one hosted VM share CPU steal,
throttling, and I/O contention. Keep the raw reports as artifacts, pass only
a bounded retry decision between jobs, and make missing or same-allocation
Expand All @@ -98,6 +118,25 @@ calibration environment.
budget. Replacing it with a real lazy workbench should initially fail only
that route's owned leaves; do not reinterpret the placeholder budget as a
calibration result.
- A schema-valid pinned measurement report can fail enforcement because a new
route's owned budget leaves are missing. Verify its exact raw-report hash,
runner, commit, schema, and samples before using it to calibrate only those
leaves with the frozen formula; an operationally invalid report is not
calibration evidence. A hard `budget_missing` failure suppresses runtime
confirmation, so other first-pass crossings remain unconfirmed. Run ordinary
enforcement again after calibration instead of widening existing budgets or
treating the completed measurement as a green gate. PR #180's Settings-only
correction is recorded in `docs/design/settings-heap-calibration.json`.
- Process isolation can deliberately trade small-job latency for responsiveness
and shorter compiler-memory lifetime. Confirm the regression on independent
pinned runners, distinguish peak aggregate RSS from surviving-parent RSS, and
record explicit acceptance of the product tradeoff before recalibration.
Use the first healthy report and existing formula for only the accepted,
confirmed timing leaves; restore their old values in the frozen-budget hash
projection so every unrelated limit stays protected. A local same-code
comparison explains the cost but never supplies release calibration numbers.
PR #180 retains both runner attempts in
`docs/design/graph-maintenance-timing-calibration.json`.

## Verify

Expand All @@ -120,6 +159,10 @@ raw samples show a real regression or stable shift.

## Update Scaffold

The 2026-09-09 revision records valid calibration versus successful enforcement,
and the explicitly accepted startup cost of graph process isolation. Existing
grounding fingerprints and `bodyHash` baselines are retained unchanged.

- [ ] Update `.mex/ROUTER.md` when the benchmark surface or pinned runner changes
- [ ] Update `docs/design/release-performance-baseline.md` with the retained calibration
- [ ] Extend this pattern when a new resource class or bundler trap is discovered
35 changes: 33 additions & 2 deletions .mex/patterns/safe-graph-snapshot-evolution.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ edges:
condition: "when changing the graph data plane or its consumers"
- target: "context/conventions.md"
condition: "when verifying a graph implementation change"
last_updated: 2026-09-07
last_updated: 2026-09-09
mex:
id: mx_01M1M0CJP81C590FCKTSN5HA3Q
type: pattern
status: promoted
revision: 3
revision: 4
title: safe-graph-snapshot-evolution
grounds_to:
- node: function:57e8797d70bfb28e3f0bb1d6e065a84b
Expand Down Expand Up @@ -82,6 +82,17 @@ only to explicit maintenance workflows.
10. Normalize evaluator provenance field-by-field. Exclude only explicitly
operational snapshot fields; malformed or future snapshot shapes must fail
closed instead of disappearing from the semantic graph hash.
11. Use `upsertFingerprintsInOwnedTransaction(db, entries)` only where the caller
owns complete publication rollback and lets every failure reach it. Keep
`FingerprintStore.upsert` and `upsertMany` independently atomic even if an
enclosing caller catches a failure and continues. Preserve duplicate
last-entry behavior, stable references, aliases, constraints, and untouched
rows. Graph publication never accepts a new Markdown grounding baseline.
12. Reuse only a fixed, owned set of synchronous storage statements on their
original connection. Keep dynamic queries and caller-owned iterators
independent; a generic global SQL cache can reuse an active iterator or
retain closed databases. Fingerprint point reads use weak connection
ownership; GraphStore hot statements belong to the store instance.

## Gotchas

Expand Down Expand Up @@ -131,6 +142,15 @@ only to explicit maintenance workflows.
- Wall-clock status timings vary by machine and process-start overhead. Keep
the benchmark non-gating, record its environment, and protect correctness
with deterministic race, non-mutation, and bounded-work tests.
- An active outer transaction does not establish rollback ownership. Omitting
the nested fingerprint savepoint is safe only for the explicit full-publisher
path. A failed multi-value bucket statement may have written a partial prefix;
standalone writes must restore the entire batch before returning an error.
- Construction in a child process does not transfer the parent's maintenance
lease or publication authority. Wait for child `close` before candidate or
workspace cleanup and recheck directory identity, including full-width
device/inode values. A parent-lifetime pipe can stop a busy child after parent
death; it cannot run cleanup in a parent killed by `SIGKILL`.

## Verify

Expand All @@ -145,6 +165,11 @@ only to explicit maintenance workflows.
- [ ] Source/config symlink escape, retarget, atomic replacement, and ABA tests
preserve the prior snapshot.
- [ ] Failed parse/stage/publication tests preserve prior facts and metadata.
- [ ] Fingerprint foreign-key and partial bucket failures restore complete
batches when caught inside an outer transaction; publisher failures roll
back all graph facts. Duplicate/ref/alias output remains identical.
- [ ] Reused statements remain correct after rollback and while independent
iterators are active; closing one connection never affects another.
- [ ] Candidate replacement, candidate WAL, rollback, maintenance-lock, and
first-publication failure tests leave either the prior graph or no graph.
- [ ] Ordinary check, doctor, dashboard, and status paths do not change graph
Expand All @@ -166,6 +191,12 @@ fix that boundary instead of adding timing delays.

## Update Scaffold

The 2026-09-09 update records the owned-transaction and statement-lifetime
contracts from the branch implementation. Existing `grounds_to` fingerprints and
`bodyHash` values are retained unchanged; this upkeep does not authorize baseline
renewal. See `docs/design/code-graph-performance-implementation.md` for evidence
and remaining process/memory limits.

- [ ] Update `.mex/ROUTER.md` when freshness, refresh, or recovery capabilities
move from "Not Built" to "Working".
- [ ] Add new graph failure modes to this pattern after they are reproduced and
Expand Down
Loading
Loading