Reduce graph refresh cost and isolate Hub construction

[theDakshJaitly]

What

Graph refresh can block the Hub while rebuilding the repository and spend minutes persisting fingerprints. Candidate construction now runs in a disposable Node process. The Hub keeps serving requests and owns the maintenance lease, validation, cancellation, and atomic publication of the last trustworthy graph.

• Use the existing outer graph transaction for internal fingerprint publication while retaining independent rollback for ordinary fingerprint batches.
• Reuse fixed connection-owned SQL statements and avoid duplicate old-node/reference staging.
• Bound private progress/result messages, validate child exit, detect parent death, and clean up only identity-verified workspaces. Jobs show files parsed without treating parsing as whole-job completion.
• Remove redundant full-database hashes while preserving exact validation/publication checks; skip unused tree-sitter startup on compiler-only extraction.
• Measure Hub plus descendants and observe maintenance completion through the production job event stream.
• Complete the previously missing Settings heap limits and explicitly accept the measured small-job startup cost by recalibrating only five confirmed Graph timing limits. Memory and every other existing limit, fixtures, formulas, and confirmation rules remain unchanged.

CLI construction remains in process. Public declarations, graph schema, and grounding baselines are unchanged. Telemetry remains separate. This draft targets codex/0.8.1; nothing is merged.

Evidence and tradeoffs

Related to #140 and the Hub refresh investigation. This does not claim reproduction or resolution of the reporter's private repository.

On the same frozen 723-file MEX corpus, the baseline engine took 305.712 s and this implementation took 71.046 s, about 4.3× faster in one ordered local pair. Native statement preparations fell from 426,415 to 117. All 12 normalized tables and four ranked search queries matched; integrity and foreign-key checks passed.

The actual Hub completed an edited-source refresh in 92.107 s. Job polling p95 was 3.944 ms, and cancellation was acknowledged in 8.036 ms while preserving the prior graph. Combined sampled RSS still peaked around 1.9 GiB, and synchronous initial/validation/publication boundaries caused pauses up to 5.238 s. There is no peak-memory quota; fatal parent termination can leave owned temporary artifacts.

Process isolation has a real startup/memory cost. An identical-code local four-file diagnostic measured 879.5 ms per isolated rebuild versus 220.2 ms in a warm process. Aggregate peak RSS increased 321 → 404 MiB, while surviving-parent RSS decreased 320 → 164 MiB. This bounded Mac diagnostic explains the tradeoff; it is not calibration evidence or a universal memory guarantee.

Two independent pinned Linux runners confirmed five small-fixture Graph timing regressions. Their explicitly accepted limits use the first healthy corrected report and the existing ceil(p95 * 1.15) formula: small refresh/rebuild 1634/1689 ms, medium 1842/1820 ms, large rebuild 2278 ms. Large refresh and all memory limits remain unchanged. Prior limits, both raw sample sets, runner identities, hashes, and unowned-budget guards are retained.

See implementation and verification, large-repository measurements, isolation diagnostic, and pinned timing calibration.

Validation

• Typecheck, build, evaluator, packed-install smoke, asset gate, focused storage/compiler/continuity/maintenance tests, and all 428 Hub web tests passed for the implementation.
• Corrected head 4d6683e passed full Node 22/24, browser, and Windows/macOS portability CI. Its two-run performance gate confirmed only the five timing failures above; repeated memory crossings remained advisory under the unchanged rules.
• Final calibration verification passed 71 focused budget/orchestration/workflow/measurement tests and all typechecks. Fresh CI run 34291831733 passed all Node, browser, Windows/macOS, and enforcing performance jobs at c93f09d. The independent performance allocations measured the same synthetic merge cdfb07300a91f0b76771281e0acdeba6ed2110a3 and produced no final violations. Large Wiki rebuild prompted confirmation, then measured 628.167 ms with zero samples above its existing 709.55 ms material threshold; its 617 ms budget remains unchanged.
• The real checkout's graph remains untouched; all benchmark writes used isolated fixtures.

Checklist

• No breaking public API, graph identity/schema, or drift-semantics changes
• Output parity, rollback, cancellation, cleanup, and responsiveness verified
• Real Windows/macOS lifecycle coverage passed on the corrected implementation
• Clean enforcing release CI on final calibrated head c93f09d (run 34291831733)
• Targets main — intentionally targets codex/0.8.1 under the release plan