docs(perf): require Bun comparisons and track mixed workload regression - #25
Merged
Conversation
huacnlee
added a commit
that referenced
this pull request
Sep 10, 2026
Short mixed-argument calls and monomorphic property access repeatedly cross runtime boundaries. Add a narrow Int32/Bool compiled-to-compiled ABI for pure forward-branch leaves, and emit native property guards using an ABI-described, non-reused shape generation token. Rooted property hits avoid the C guard and eager frame publication; guard misses reconstruct the original bytecode state. Also admit mixed numeric array loops to existing guarded automatic Tier 2 trials, reduce cold metric publication, and cache safe resident artifact lookups. Paired fixed-warmup results against the PR #25 runtime show **26.20x generic-call-entry, 6.91x property-heavy, and 2.65x arrays-typed** speed, respectively **3.99x, 3.17x, and 1.64x** their same-version QuickJS interpreter. The call gain combines enabling the compiled caller and direct leaf ABI; it is not an isolated generic-call cost comparison. A new effectful `generic-call-fallback` keeps the non-direct/native-entry workload visible: it improves 1.205x over the old JIT but remains only 0.339x QuickJS speed. These three priority workloads still trail default Bun substantially. The README reports every one of the 24 QuickJS/Bun/automatic-JIT scenarios, including slower, tied, and fallback results. The shared driver applies equal warmup, consumes every result, and checks checksums outside timing. Raw samples, immutable source snapshots, binary hashes, paired confidence intervals, and host measurements are archived. This measures fixed-warmup batch latency, not peak Bun or matched settled-state throughput. ABI minor advances to 21 with a checked PropertyLayout descriptor. On the measured x86_64 build, JSShape grows from 64 to 72 bytes and JSRuntime grows by 8 bytes; JSFunctionBytecode stays at 160 bytes. The earlier PC-cache experiment was removed after host regression isolation. Numeric coercion, overflow, target replacement, shape/descriptor mutation, ownership, and exact deoptimization paths retain guarded recovery. Full LICM, bounds-check elimination, general inlining, and closure/async ABI expansion are outside this change. Validation: Exact final source passed 52 release semantic/differential/GPUI-surface tests and 340 C/Rust AddressSanitizer tests (generated JIT machine code itself is not ASAN-instrumented). The pinned historical GPUI Shell harness passed 10 tests with 1 ignored. A compiler-only production_api trybuild diagnostic mismatch reproduces on baseline and candidate; this check is not reported as passing. Scalar and call-heavy paired controls are statistically tied with the old JIT; their confidence bounds satisfy this run's 0.95x speed budget. This does not establish every M2 gate. **Draft: host regression remains unresolved.** Final mixed-host steady snapshots are 1.154x [1.146, 1.161] the old JIT speed, but compute hot reload is only 0.944x [0.937, 0.952] and fails the 0.95x budget. Its interpreter control is 0.933x [0.930, 0.935] in steady snapshots. Compute automatic steady state is statistically tied (between 11.1% slower and 19.8% faster). The final regression is not causally attributed to the removed PC cache; investigation is still needed before merge. All final raw observations are retained without additional confirmation rounds. Refs #14. Follows #25. --------- Co-authored-by: Codex <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes
Require every JIT optimization to include QuickJS, Bun default, and production automatic quickjs-jit benchmarks, with the complete per-scenario matrix in the root README. Replace the three selected forced-Tier2 examples with all 22 measured scenarios, including slow and fallback cases, timings, speed ratios, and confidence intervals.
The README explicitly labels these results as the historical
47aeb11matrix. Current07535b1has not been remeasured against Bun across the full matrix. The next-stage goals prioritize Bun-relative gaps and require aligned warmup and timing boundaries before establishing the next baseline.Also archive the independent Linux x86_64 gpui-shell mixed-workload regression investigation: 30 retained interleaved processes per configuration reproduce 0.700x the old JIT's speed (95% CI 0.694–0.707x), with approximately 0.059 ms additional latency per snapshot. Sixty profiler runs identify sorting callbacks and short native-call overhead as investigation targets, not a confirmed root cause or fix. The archive retains original/recheck samples, scripts, profiler experiments, build provenance and dependency-resolution caveats.
Validation
git diff --cached --check.Checklist