diff --git a/STATUS.md b/STATUS.md index 91107be..6cb9c91 100644 --- a/STATUS.md +++ b/STATUS.md @@ -11,6 +11,30 @@ out, and no cron trigger is active. Nothing is half-finished: every line is written up, and the queue below is a list of starting points rather than abandoned work. +**2026-08-06, one line (attempt 018, skeptic-confirmed 019): Gap 1's +margin-modulated candidate is REFUTED in both signed readings at n = 7 — +by 007's own 10-atom witness, the instance it was invented to survive.** +The h-sensitivity weighting of 007 lead 2 splits into three precise +readings; the assembly-exact secant form is certified negative +(−0.02225465820566 at the exactly-rational tilt t = 181/16, in-regime, +dichotomy exact — the skeptic's from-scratch engine, different census and +different enclosure kit, matches to width 5.9e-30) on the same tables +where the plain aggregate is certified +1.844669. Mechanism: the +sensitivity weight is signed, and in the record regime the surplus +histories' realized both-zero probabilities sit past the entropy peak +(z̃ > 1/2), so the modulation flips the aggregate's biggest positive +terms — it kills the surplus, not the deficit it was designed to +down-weight. What survives of Gap 1: the unsigned |σ|-weighted control +(positive on everything tried, including 016's three certified kill +instances; softest point +0.0101 at a free-support n = 8 endpoint) and +the λ-window-restricted control (positive across the θ-re-optimized +ladder genre to n = 320, with the first-order protection a₁(n)·n GROWING +4.7 → 18.1 and the joint (θ, λ) attack converging to the trivial λ → 0 +boundary). Also settled: 016 lead 3 — the raw-weight ladder does NOT +cross through n = 320. Five corrections in 019, two substantive (raw +ladder MM signs flip on the n ≥ 96 dilution branch; a wrong golden-ratio +aside), neither load-bearing. + **2026-08-04, two cycles.** First (Astra follow-through): a literature-novelty check joined the skeptic pass, the formalization lane (`docs/FORMALIZE.md`, status `FORMALIZED`) went in above `VERIFIED`, the @@ -132,7 +156,7 @@ problems with no attempts (queue 18–19; run blind). | Problem | Status | Budget | Active line | |---|---|---|---| | Erdős–Gyárfás | n=24 done | high | next: 2-connected C16-free test (lead 2); n=26 is a ~16h run | -| Union-closed (Frankl) | ROUTE LIVE (verified) | high | gap (a′) refuted (007/013); AGGREGATED control certified positive to n = 32 (014/015) but refuted at large n (016/017, certified n = 96–160) → live: margin-modulated control, λ ≲ c/n restricted variant, δ-linear tax budget, corrected assembly budgets (012); plus three sweep leads (010) | +| Union-closed (Frankl) | ROUTE LIVE (verified) | high | gap (a′) refuted (007/013); AGGREGATED control refuted at large n (016/017); margin-modulated control refuted in both signed readings at n = 7 (018/019, certified) → live: unsigned \|σ\|-control (attack it at free supports n = 10–16, 018 lead 1), λ ≲ c/n window variant (survives the ladder to n = 320; hunt a₁-degenerate near-product families, 018 lead 4), assembly-requirement restatement (018 lead 2), δ-linear tax budget + corrected assembly budgets (012); plus three sweep leads (010) | | Erdős–Straus | 601 resolved | medium | next: prove the identity-poor mechanism (why QR classes force low f) | | Singmaster | census done | medium | next: Diophantine curve table (search-deeper is now low value) | | Lonely runner | k=8 done | medium | next: k=9 scan (k=8 likely settled by Rosenfeld preprint) | @@ -147,7 +171,7 @@ problems with no attempts (queue 18–19; run blind). ## Attempt queue (next cycles pull from the top) -1. [union-closed] ~~Probe the aggregated control at n ≳ 20~~ **DONE twice, independently, in 014/015 and 016/017** — positive to n = 32 with the margin growing, then negative at n = 96–160 on the MU(n,r) unit ladder. The ∀λ aggregated control is dead; the boundary-positivity program 014 proposed is off for it. The surviving Gap-1 candidates, probe-first per standing policy: (i) **margin-modulated OR control** — run the certified probe on the MU(n,r) ladder genre FIRST (it is the reusable adversary; 017's independent engine `uc_agg_ctrl_skeptic.py` certifies), then the killer genres from 007/013 and 014's battery (a one-line weight change there); only if it survives n ≳ 96 does proof effort start (007's Gram/Frobenius machinery). (ii) **λ-restricted variant**: the control restricted to the workable window λ ≲ 4.847/(n−3) (the 009/011 λ-window law) — 016's violation lives at λ ∈ [2, 2.5] and its family is positive at λ ≤ 1.5, so this variant is genuinely untested; probe the ladder AT the window boundary across n before anything else. Two structural questions from 014/015 keep their value under either candidate and are cheap: settle the equality-set converse for λ > 0, and map the near-zero slice-direction dip (+1.4e-7 at n ≈ 48, recovers by 64) over p × layer-ratio × n ≤ 96. Mind 015's R1 (fitted orders dip to d^1.39 at λ ≥ 0.5), so no Hessian-style argument gives a finite-d positivity radius on its own. Any claimed bound → skeptic review before ledger entry. +1. [union-closed] ~~Probe both surviving Gap-1 candidates against the ladder adversary~~ **DONE in 018/019** — (i) the margin-modulated control is REFUTED in both signed readings at n = 7 by 007's own witness (certified −0.0222546582 at t = 181/16; the sensitivity weight is signed and kills the surplus, not the hidden deficit), and (ii) the λ-window-restricted control SURVIVES the θ-re-optimized ladder to n = 320 with a₁(n)·n growing. The replacement program, probe-first: (a) **attack the unsigned |σ|-control at free supports n = 10–16** seeded from 018's +0.0101 endpoint (`gap1c_partC.json` best_free, 2500+ steps, atom-count mutations) — kill it or find its floor; (b) **hunt a₁-degenerate families for the window variant** (018 lead 4): near-product families along 014's slice-direction dip scaled with n, evaluated at λ_win(n) = 4.847/(n−3) — products are a₁'s equality set, so this is the window candidate's most dangerous direction; (c) **restate what the assembly actually needs** now that the per-history secant form is certified negative on the witness (018 lead 2 = 016 lead 5): derive the coordinate-level or λ-integrated inequality the 008/012 assembly requires and test it with 018's census engine. If (a) and (b) both come back positive, first-order proof effort starts with 018 lead 3 (closed-form lower bound on the ladder's a₁). Two cheap structural questions from 014/015 keep their value: the equality-set converse for λ > 0, and the slice-direction dip map. Any claimed bound → skeptic review before ledger entry. 2. [union-closed] Rebuild the assembly budgets per 012's corrections: the tax is δ-LINEAR (restate B2), the τ_half step needs s₀ ≤ 0.0843, corrected δ₀ ≈ 0.004. The open question is whether ANY n-uniform budget object exists — 008's conditional theorem survives structurally; its constants need the corrected inputs, and the budget census machinery (uc_pert.py + uc_pert_skeptic.py's orbit engine) is ready. 3. [union-closed] Sweep leads from 010, one per cycle: (i) pairwise-closure LP/degree-2 SOS certification on the n ≤ 4 census (all 4958 families) — kill: certified bound converges to ≈ 0.382; (ii) union-transfer-operator eigenvalue field — test log-supermodularity of λ_C on the n ≤ 5 census (pre-check the total-positivity records first); (iii) bipartite-MIS decomposition — geng census to n = 12, connectivity of extremals, compositionality across 1- and 2-cuts. Kill conditions recorded in 010. 4. [erdos-straus] Prove the identity-poverty mechanism: why does QR-class membership mod 840 force fewer Type I covering congruences? Start from 001's obstruction analysis + 002's rate data; target a theorem "f(p) ≥ g(N_typeI(p))" or a disproof. @@ -166,7 +190,7 @@ problems with no attempts (queue 18–19; run blind). 17. [maxwell-equilibria] Three-positive-charge census hunting the open 4-vs-6 gap: after 15's strata map, target the strata boundaries (where counts jump) with unequal charges. Win condition kept in view by every run: any configuration certified with ≥ 5 isolated equilibria refutes the conjectured max 4 and is a result people have sought since 1873 — it must survive verify_equilibria.py and be reported as requiring escalation. Equal-magnitude configurations are settled (Tsai 2015, max 4) — spend no effort there. 18. [almost-mathieu] **Run blind.** Rational-flux gap census, all p/q with q ≤ 30: certify every gap open except the even-q central touching (re-derives van Mouche / Choi–Elliott–Yui in range; expected `VERIFIED`, scope = the q range), then the golden-mean convergent table — exact minimal-gap widths and q·|σ| along Fibonacci p/q as far as tooling reaches, against the (unproven) Thouless constant 32C/π. Onboarding smoke runs: q·|σ| = 9.2509 / 9.3199 / 9.3608 at q = 13 / 21 / 34 vs 9.3299 conjectured. Every record re-verified with `verify_bands.py` before ledger entry. Kill condition: if exact arithmetic stalls before q ≈ 100 even with a C kernel to the same contract, record the wall — no asymptotic claims from small denominators. 19. [three-phase-conductivity] **Run blind.** Two-phase ground truth first: rank-2 laminates attaining the 2D HS bounds exactly in ℚ, duality checks, series/parallel forms (expected `VERIFIED`, harness validation). Then the three-phase attainability map: fixed rational (σ₁,σ₂,σ₃), rational grid on the fraction simplex, bounded-rank laminate optimization (float screen, exact certification), gap-to-HS charted per cell (`MAP`/`EVIDENCE`, scoped by rank + direction set + grid). Nesi/Cherkaev improved bounds enter as marked transcriptions cross-checked against the papers' examples before anything is killed against them. Kill condition: bounded-rank optima plateauing strictly inside bounds across the whole grid = one negative-map record, then cap the budget. -20. [union-closed] **Push the kill's frontier** (cheap, from 016 leads): direct θ-optimization of the MU ladder at target n to find the minimal violating n (currently bracketed (32, 96]); extend the raw-weight ladder past n = 128 (certified +0.157 at 96 — does it also eventually cross?); and re-run the 016 ladder at the λ-window boundary as the first data point for queue item 1(ii). +20. [union-closed] **Push the kill's frontier** (cheap, from 016 leads): direct θ-optimization of the MU ladder at target n to find the minimal violating n (currently bracketed (32, 96]). ~~Extend the raw-weight ladder past n = 128~~ and ~~re-run the ladder at the λ-window boundary~~ **DONE in 018** — the raw ladder does NOT cross through n = 320 (decaying along each dilution branch, min +0.104 at n = 256; the 016 kill is entirely the re-weighting), and the window boundary is positive at every n ≤ 320 tried, θ re-optimized there included. ## Verified results @@ -545,8 +569,59 @@ problems with no attempts (queue 18–19; run blind). equality-set structure is what makes the λ-restricted variant the live question. +- **[union-closed] Margin-modulated OR control REFUTED in both signed + readings at n = 7 (skeptic-confirmed)** (2026-08-06, attempts 018+019): + 007 lead 2's "h-sensitivity-weighted" candidate splits into three + precise statements; the assembly-exact secant form + E[h₂(z̃) − h₂(z_{2^λ}(x̃, ỹ))] and the derivative-at-target form are + both negative on 007's own 10-atom witness — the instance the weighting + was invented to survive — with the secant form certified float-free at + t = 181/16 (MM_sec = −0.0222546582…, in-regime, dichotomy exact, the + first certified enclosure of a nonlinear census functional in the + library: dyadic-bisection Plackett roots + directed-log₂ binary + entropy, margin-pair cached). 019 re-certified from scratch (own + census from 007 §1's prose, exact-rational Newton with symbolically + sign-checked brackets, interval-squaring log₂ — no shared code) to + width 5.9e-30, matching; re-derived the secant identity, the + product-measure zero, and the degenerate-history invariance by hand; + confirmed no natural signed variant survives (realized-OR weighting + dies too, −1.458); and newly certified the 2-digit tidy witness + (−0.0222736748). Perturbation-stable (20/20 at 3%, plus 40/40 on the + skeptic's fresh seeds), violating for every λ ∈ [1, 5] (crossing + λ ≈ 0.83, 019 C4). Mechanism: the weight is signed and the surplus + histories sit past the entropy peak. Survivors, EVIDENCE-scoped: the + unsigned |σ|-weighted control (positive everywhere tried incl. 016's + certified kill instances; θ re-optimized against it, transfers RISING + with n; softest +0.0101 at a free n = 8 endpoint — only 3 of 8 free + climbs ended in-regime, 019) and the λ-window-restricted control + (positive at λ_win, λ_win/2, λ_win/4 across raw and θ\* ladders + n ≤ 320, θ re-optimized at window λ, joint (θ, λ) climb converging to + the trivial λ → 0 boundary; a₁(n)·n grows 4.7 → 18.1 θ\*, ≈ 66 raw). + 016 lead 3 settled: the raw-weight ladder never crosses through + n = 320 (min +0.104 at n = 256). Corrections of record (019): raw + ladder MM signs flip positive on the n ≥ 96 dilution branch (018's + "negative at every n" is θ\*-only); the golden-ratio aside is false + as stated (ρ = 1 crossing at marginal 1 − 2^{−1/2}, not (3−√5)/2); + witness MM_sec crossing λ ≈ 0.83 not ≲ 0.7; §F certificates landed + after review close (not covered by 019); runtime understated. + ## Insights / cross-problem notes +- Signed sensitivity weights backfire (2026-08-06, union-closed 018/019): + weighting a deviation control by the derivative of the gain it feeds + does not neutralize an adversary that hides its deficit at degenerate + margins — the derivative is SIGNED, and in the record regime the + surplus histories sit on the decreasing side of the entropy curve, so + the modulation flips the control's biggest positive terms instead of + suppressing the deficit. Salvaging positivity needs the unsigned + weight, which decouples the functional from the chain rule it was + meant to serve — a modulated control can be plausible-looking and + assembly-irrelevant at the same time. Same cycle, tooling note: a + NONLINEAR census functional (algebraic root + entropy per history) is + certifiable at ladder scale by enclosing the root with dyadic + bisection and caching by the margin pair — unit exchangeability + collapses thousands of rows to a handful of enclosures. + - Blind mode, data point #3 (2026-08-04, crouzeix 001/002): with prior art physically absent, the census blind-rediscovered both known extremal structures (the ratio-1 ice-cream-cone configuration and the diff --git a/problems/union-closed/attempts/018-surviving-controls-vs-ladder.md b/problems/union-closed/attempts/018-surviving-controls-vs-ladder.md new file mode 100644 index 0000000..ff80433 --- /dev/null +++ b/problems/union-closed/attempts/018-surviving-controls-vs-ladder.md @@ -0,0 +1,423 @@ +# 018 — The two surviving Gap-1 candidates vs the ladder adversary + +- **Problem:** union-closed, `problems/union-closed/PROBLEM.md` +- **Date:** 2026-08-06 +- **Mode:** informed +- **Type:** computational search + candidate-statement triage (queue items 1 + and 20; 016 leads 1 and 4). Outcome: the **margin-modulated control is + REFUTED in both signed readings at n = 7** — by 007's own witness, the + instance it was invented to survive — certified float-free; the |σ|-weighted + unsigned reading and the **λ-window-restricted control survive** the full + ladder adversary with re-optimized weights (`EVIDENCE`, scoped below). +- **Tools:** `explore/uc_gap1_candidates.py` (written here; standard-library + only, deterministic — fixed seeds, fixed step counts). Parts A–D (default, + ~35 min), `--ext` (part E, ~25 min), `--cert` (part F, exact rational + certification), `--robust` (part R, kill-robustness battery), `--fast` + (smoke). Float census logic is a fresh implementation cross-anchored + against 007's engine (`uc_or_avg.py`, imported for the plain aggregate); + exact machinery is 013's (`uc_or_avg_skeptic.py`) plus a new certified + interval path for the margin-modulated quantity (dyadic bisection for the + Plackett root, directed log₂ enclosures for the binary entropy). Commands + reproducing every number: + + python problems/union-closed/explore/uc_gap1_candidates.py 2>&1 | tee problems/union-closed/data/gap1c_run.log + python problems/union-closed/explore/uc_gap1_candidates.py --ext 2>&1 | tee problems/union-closed/data/gap1c_ext.log + python problems/union-closed/explore/uc_gap1_candidates.py --robust 2>&1 | tee problems/union-closed/data/gap1c_robust.log + python problems/union-closed/explore/uc_gap1_candidates.py --cert 2>&1 | tee problems/union-closed/data/gap1c_cert.log + + Checkpoints: `data/gap1c_part[A-F,R].json`. +- **Sources:** 007 (lead 2 — the margin-modulated candidate; §1 M_i + definition; the 10-atom witness), 013 (exact-arithmetic standard, aggregate + certification), 016/017 (the MU(n,r) ladder, θ\*, the λ-window + disjointness correction C2), 009/011 (the λ-window law + λ_max ≈ 4.847/(n−3)), 008 (h(z_ρ) gain machinery). No external fetches + this cycle. + +## Approach + +**Context.** 016/017 killed the i-aggregated OR control at n ≥ 96 and left +exactly two Gap-1 candidates standing: the margin-modulated control (007 +lead 2) and the λ-window-restricted control (017 C2). The queue's standing +policy is probe-before-proof, and 016 supplies the one adversary genre known +to kill at scale: the unit-replicated ladder MU(n, r) with adversarially +re-optimized shared weights θ. This attempt runs that adversary — with θ +re-optimized against each NEW statement, since a survival against weights +tuned for a different functional would be worthless — against both +candidates, plus free-support and (θ, λ)-joint attacks. + +**Making candidate I precise (the fork in 007 lead 2).** "E[σ(x̃,ỹ)·(log₂OR +− λ)] ≥ 0 with σ the Plackett sensitivity" admits three readings, which this +attempt formalizes and tests separately. At a nondegenerate history (a, b) +of coordinate i, write the conditional table's zero-margins x = P(A_i = 0), +y = P(B_i = 0), realized both-zero probability z̃, and let z_ρ(x, y) be the +Plackett both-zero probability at odds ratio ρ (the unique root of +z(1−x−y+z) = ρ(x−z)(y−z) in the coupling range). Then, mass-weighted over +all nondegenerate histories of all i < n: + +- **MM_sec** (secant / assembly-exact): E[h₂(z̃) − h₂(z_{2^λ}(x, y))]. By + the mean-value theorem this IS E[σ·(log₂OR − λ)] with σ the sensitivity + ∂h₂(z_ρ)/∂λ evaluated on the secant — and h₂(z̃) = h₂(z_OR(x, y)) exactly, + since a 2×2 table is determined by its margins and odds ratio. This is + the reading with chain-rule meaning: it compares each history's realized + Plackett gain against the gain the λ-target would deliver. +- **MM_der** (signed derivative at target): E[σ_λ·(log₂OR − λ)] with + σ_λ = ∂h₂(z_ρ(x, y))/∂λ at ρ = 2^λ. +- **MM_abs** (unsigned weight): E[|σ_λ|·(log₂OR − λ)], normalized by + E[|σ_λ|] — a weighted mean of (log₂OR − λ) with a nonnegative weight that + vanishes as the margins degenerate, which is the literal "downweight the + witness's hiding place" reading. + +The distinction matters because σ_λ changes sign at z = 1/2, and in-regime +margins straddle that point (z_1(x, x) = x² crosses 1/2 exactly at the +(3−√5)/2 barrier — the sensitivity sign boundary is the record threshold's +golden-ratio structure again). + +**Candidate II** is the plain aggregate A(μ, λ) of 014/016 restricted to +λ ≤ λ_win(n) = 4.847/(n−3) (the 009/011 window law — the regime the +assembly can actually use at large n). 016's kill lives at λ ∈ [2, 2.5], +~40× above the window at n = 96 (017 C2), so this candidate was genuinely +untested. The probe measures the ladder INSIDE the window with θ +re-optimized at window λ, lets the adversary choose λ within the window, +and fits the small-λ expansion A ≈ a₁(n)λ + a₂(n)λ² — the structural +question being whether a₁(n) (Theorem C's perfect-square first-order +coefficient) decays fast enough for the quadratic term to cross inside a +window that itself shrinks like 1/n. + +**Why this rather than proof effort on either candidate:** 016 is the +second demonstration (after 012) that small-n survival licenses nothing; +both candidates had zero adversarial exposure in their stated regimes. + +## What was done + +### A. Anchors (all pass; `gap1c_partA.json`) + +Witness plain aggregate reproduces 013 (+1.844754 float at λ = 3.5, and ++1.844669005 exactly in the part-F certification); the fresh census's +internal plain aggregate matches 007's engine to 2.2e-16; 016's θ\* kill +instance reproduces (−0.000759865 at n = 96, λ = 2); the Plackett +round-trip z_ρ(x, y, OR) = z̃ holds to 1.6e-15 on every witness row (the +2×2-table-determination fact the secant identity rests on); secant→derivative +consistency at Δλ = 1e-3 (rel 2e-4); MM padding invariance MU(20,1) ≡ +witness, diff 0.0 exactly. Product-measure anchor (part R): MM_sec vanishes +identically on Bern(p)^n (|MM_sec| < 3e-16; OR = 2^λ at every history so +z̃ = z_target row by row — an exact structural zero the float path +reproduces). + +### B. Candidate I on the witness and the ladders (`gap1c_partB.json`) + +The signed readings fail immediately, and not at large n: + + 007 witness (n = 7, λ = 3.5, marginals ≤ 0.318): + plain aggregate +1.8448 (the value 013 certified positive) + MM_sec −0.022255 VIOLATION + MM_der −0.895542 VIOLATION + MM_abs +2.354328 survives + +λ-profile on the witness (part R): MM_sec < 0 for every λ ∈ [1, 5] +(minimum −0.0224 near λ = 4), positive only for λ ≲ 0.7 — exactly the +first-order-dominated region where Theorem C forces positivity. MM_der +flips negative from λ ≈ 1.5. So the margin modulation does not rescue the +control on the very instance that killed the unmodulated version — it makes +it fail *harder* (the plain aggregate is +1.84 there; the modulated ones +are negative). + +On the ladders at λ = 2 (raw witness weights and 016's θ\*), MM_sec is +negative at every n ∈ {16, …, 160} (θ\*: −2.4e-3 at n = 16 shrinking to +−2.2e-4 at n = 160; the replication that kills the plain aggregate +*shrinks* the MM_sec deficit), MM_der negative throughout, MM_abs positive +throughout (≈ +0.97 on θ\*, including on the three certified kill +instances of 016). Sign context (part R): MM_sec is NOT generically +negative — it is +0.0004…+0.006 on Bernoulli mixtures, smoothed slices, +and random dense potentials, ≈ 0 at products, and negative on the +witness/ladder genre and (out-of-regime) on δ₀⊕Bern(0.54) at λ = 1. The +violation is structural to the two-block geometry, not a Jensen artifact +of the definition. + +### C. The adversary vs MM_abs (`gap1c_partC.json`) + +θ re-optimized against MM_abs (260 steps, marginal-penalized, n = 48, +λ ∈ {2, 3.5}), then transferred to n = 64…160 with per-n dilution re-tune; +free-support climbs at n = 8 against MM_abs (8 × 600 steps, witness-seeded +and random-seeded, penalty 300). **No violation of MM_abs anywhere:** + +- θ-opt at λ = 2 ground MM_abs from ≈ +0.97 (θ\*) to **+0.246** at n = 48 + — a 4× margin cut — but the transfers RISE with n: +0.239 (64), +1.004 + (96), +1.055 (128), +1.094 (160). At λ = 3.5 the optimizer reached + +0.064 at n = 48, transfers again rising to +0.159 at n = 160. Unit + replication, the move that kills the plain aggregate, does not scale the + |σ|-weighted deficit — the weight suppresses exactly the near-degenerate + light-slice channel the ladder replicates. +- Free-support climbs at n = 8: tightest in-regime endpoint **+0.0101** + (witness-seeded, λ = 2, max marginal 0.153) — a ~100× margin cut vs the + ladder values, still positive; the other seven endpoints +0.16…+5.9. + MM_sec at those endpoints ranges −8.7e-3…+8.8e-2 (both signs — the + free search finds the signed violation trivially, consistent with part B, + while the unsigned form resists). + +### D. Candidate II inside the window (`gap1c_partD.json`) + +Ladder scans at λ = λ_win(n), λ_win/2 (and λ_win/4 for n < 224) for θ\* +and raw weights, n ∈ {48, 96, 160, 224, 320}; θ re-optimized AT window λ +(n = 48 and 96, 300 steps), transferred with per-n window λ up to n = 320; +joint (θ, λ ≤ λ_win) climb at n = 96; small-λ coefficient fits. +**Every window point positive, with the margin structure strengthening in +n:** + +- θ\* ladder at λ_win(n): +1.01e-2 (48), +3.75e-3 (96), +1.94e-3 (160), + +1.29e-3 (224), +8.52e-4 (320). The fitted expansion + A ≈ a₁λ + a₂λ² has a₂ < 0 (the crossing direction) but **a₁(n)·n + GROWS**: +4.7 (48) → +7.2 (96) → +10.4 (160) → +13.5 (224) → +18.1 + (320), while |a₂| stays ≈ 0.06. At λ = c/n the first-order term a₁·c/n + beats the quadratic a₂c²/n² unless a₁·n → 0; it is doing the opposite. + Raw weights: same shape, a₁·n reaching ≈ 66. +- θ re-optimized AT window λ: best the adversary managed is +5.49e-3 + (n = 48, λ = 0.1077) and +1.02e-3 (n = 96, λ = 0.0521); both θ's + transferred with per-n window λ stay positive through n = 320 + (+3.01e-4 and +1.97e-4). +- Joint (θ, λ ≤ λ_win) climb at n = 96: the optimizer converged to the + λ → 0 boundary (endpoint +1.66e-5 at λ = 0.0008) — it can shrink A only + by shrinking λ toward the trivial zero, exactly the signature of + first-order (Theorem-C square) dominance inside the window; no interior + negative found. + +### E. Raw-weight ladder extension, 016 lead 3 (`gap1c_partE.json`) + +Honest (dilution-swept minimum) raw-witness-weight ladder at λ = 2: ++0.190 (160), +0.150 (192), +0.123 (224), +0.104 (256), then a dilution +feasibility-branch jump (dil 80 → 160, as in 016 P6's n = 48/128 +artifacts) to +0.214 (288), +0.181 (320). **No crossing through n = 320**: +the raw ladder decays along each dilution branch but stays positive — the +016 kill really does need the re-optimized θ, quantifying how much of it +is re-weighting (all of it, at these n) vs pure replication. + +### F. Exact rational certification (`gap1c_partF.json`) + +The decisive point is certified float-free by a new interval path: exact +census tables (013's engine), z_target enclosed by 80-step dyadic bisection +of the exact Plackett quadratic (per distinct margin pair — the ladder's +unit exchangeability collapses thousands of rows to a handful of distinct +(x, y) pairs), h₂ enclosed via directed log₂ enclosures at dyadic interval +endpoints (straddle-guarded at z = 1/2), all sums exact rationals: + + witness, t = 181/16 (λ = log₂(181/16), exactly rational tilt): + MM_sec numerator/denominator enclosure: + MM_sec ∈ [−0.0222546582056…, −0.0222546582056…] CERTIFIED NEGATIVE + plain aggregate ∈ [+1.844669005…, +1.844669005…] (= 013's certificate + digit-for-digit, an internal anchor computed from the same tables) + max marginal < 38271/100000 exactly; degeneracy dichotomy exact. + +The remaining four jobs (completed AFTER 019's review closed — see the +scope note below): + + θ* ladder n = 96, t = 4 (λ = 2, 016's kill instance; 188 atoms, + 368 nondegenerate rows): + MM_sec ∈ [−3.819411484e-4, −3.819411484e-4] CERTIFIED NEGATIVE + plain aggregate ∈ [−0.000760, −0.000760] (= 016's kill, re-derived + from the same tables), in-regime, dichotomy exact (1436 s) + θ* ladder n = 160, t = 4 (tidy rational weights, 624 rows): + MM_sec ∈ [−2.229998913e-4, −2.229998913e-4] CERTIFIED NEGATIVE + plain ∈ [−0.024226, −0.024226], in-regime, dich exact (4069 s) + window points, t = 51/50 (λ = log₂(51/50) ≈ 0.0286, strictly inside + the window at both n), θ* window-tuned ladders: + n = 96: agg ∈ [+2.092188747e-3, +2.092188747e-3] CERT. POSITIVE + n = 160: agg ∈ [+1.802171374e-3, +1.802171374e-3] CERT. POSITIVE + both in-regime, dichotomy exact (140 s, 467 s) + +So the secant-form kill is certified not only on the witness but on the +ladder at n = 96 and 160 (the MM_sec float values of part B bracketed +digit-for-digit), and candidate II's survival has certified in-window +anchors at kill-scale n at an exactly-rational tilt. + +**Scope note (019 C3):** the independent skeptic pass (019) closed while +only the witness certificate had landed; the four certificates above are +therefore NOT covered by 019's re-verification. 019 did independently +reproduce the underlying float values (θ\* n = 96 triple, window +A(λ_win) at n = 96) with its own spec-rebuilt evaluator, but the exact +enclosures in this block await a future skeptic. The headline REFUTED +status rests only on the witness certificate, which 019 re-certified +from scratch to width 5.9e-30. + +Robustness of the kill (part R, `gap1c_partR.json`): 20/20 independent 3% +weight perturbations keep MM_sec < 0 in-regime (range −0.0228…−0.0218); +the 2-significant-digit tidy witness gives −0.0223 at max marginal 0.313. +Convention robustness is a one-line proof this time: a margin-degenerate +history has z̃ equal to its boundary value and z_target pinned to the same +boundary (z_ρ(0, y) = 0, z_ρ(x, 1) = x, etc. for every ρ > 0), so its +secant contribution is exactly 0 — including degenerate histories in the +average changes neither the numerator nor the sign, only the denominator, +exactly as in 016's robustness note. + +## Outcome + +**REFUTED (certified): the margin-modulated odds-ratio control in both +signed readings — the assembly-exact secant form and the +derivative-at-target form — fails at n = 7, in the record-relevant regime, +on 007's own 10-atom witness.** MM_sec ∈ [−0.02225465820566…, +−0.02225465820566…] at the exactly-rational tilt t = 181/16, all marginals +< 0.38271, dichotomy exact — a theorem about the stated rational inputs by +013's standard. The h-sensitivity weighting was conjectured (007 lead 2) to +neutralize the witness's light-slice hiding mechanism; instead the witness +kills the weighted statement while its plain aggregate is certified +POSITIVE (+1.8447) on the same tables. No large n, no replication, no +re-optimization was needed. + +**EVIDENCE (scoped): the two surviving statements resist the full ladder +adversary.** + +- **MM_abs** (the unsigned |σ|-weighted control): positive on everything + tried — the witness (+2.354), raw and θ\* ladders n ≤ 160 at λ = 2 + including 016's three certified kill instances (≈ +0.97), the λ-profile + at n = 96, θ re-optimized against it (floor +0.064 at n = 48, λ = 3.5, + transfers rising with n), and 8 free-support climbs at n = 8 (floor + +0.0101 in-regime). Scope: this battery only; the +0.0101 endpoint says + free supports at small n get 100× closer than any ladder — see lead 1. +- **λ-window-restricted control** (A(μ, λ) ≥ 0 for λ ≤ 4.847/(n−3)): + positive at every window point tried — ladders (both weight sets) at + λ_win, λ_win/2, λ_win/4 for n ∈ {48, 96, 160, 224, 320}, θ re-optimized + at window λ at n ∈ {48, 96} with per-n-window transfers to 320, and a + joint (θ, λ) climb at n = 96 that converged to the trivial λ → 0 + boundary. Structurally: a₁(n)·n grows (4.7 → 18.1 on θ\*, to ≈ 66 raw) + while |a₂| ≈ 0.06 stays flat, so on this genre the window shrinks + faster than the first-order protection decays. Scope: the MU(n, r) + ladder genre plus θ/λ climbs; nothing here bounds a₁ below for + arbitrary μ (see lead 4 — near-product families are where a₁ can + degenerate). + +**Not claimed:** no claim that MM_abs or the window-restricted control +holds (finite battery, one adversary genre family plus free-support climbs +at small n; 016 showed exactly how such survivals can be small-n or +wrong-genre artifacts); no claim about the dependent-couplings route +dying — the interface, licensing lemma, separations and the 0.4315 ceiling +are untouched; the MM_der violation is float EVIDENCE (−0.90 with the +certified secant identity tying the two readings; the derivative form +itself was not separately certified); no claim that the ladder is the +worst family for either survivor. Per repo rules nothing here is a result +until an independent skeptic pass; the exact MM certificate is the thing +to attack first (it is a finite list of rational-table claims plus one +dyadic bisection per distinct margin pair), then the census definition's +match to 007 §1 (anchored by the +1.844669005 reproduction), then the +search-completeness claims, which are the weakest. + +## Why it failed / what survived + +**Why the margin modulation backfires: the weight is signed, and the +witness's surplus lives on the wrong side of z = 1/2.** The sensitivity +σ = ∂h₂(z_ρ)/∂λ is positive only while z_ρ < 1/2. In the record regime the +diagonal surplus histories (the ones whose large positive log₂OR − λ the +plain aggregate spends) sit at zero-margins x, y ≈ 0.6–0.7 with z̃ close +to min(x, y) — i.e. z̃ > 1/2, where h₂ is *decreasing*: their realized +gain h₂(z̃) is LESS than the target gain, so the modulated functional +converts the plain aggregate's biggest positive terms into negatives. The +deficit history (the light slice) contributes almost nothing in either +direction — its margins are nearly degenerate, which is what 007 lead 2 +correctly anticipated — but the modulation kills the *surplus*, not the +deficit. Numerically, on the witness at λ = 3.5 the plain census has one +−0.122 deficit against several multi-bit surpluses; the secant census has +those same surpluses contributing ≈ −0.02 net. The golden-ratio +coincidence (z crosses 1/2 exactly at marginal (3−√5)/2 for equal margins) +means this sign flip is intrinsic to the record regime: any +sensitivity-weighted control inherits it, which is why the failure needs +no adversarial tuning at all. + +**What the |σ|-weighted form dodges, and what it costs.** Taking |σ| +restores positivity on everything tried — but it decouples the functional +from the chain rule: |σ|·(log₂OR − λ) is no longer the derivative of any +gain, so a proof of MM_abs ≥ 0 would not by itself feed the 008/012 +assembly. Mechanically, MM_abs resists the ladder because replication +scales a deficit that the weight has already zeroed: the per-unit deficit +history is the light slice, its margins are nearly degenerate, and +|σ| → 0 there — so the Θ(n) deficit channel that kills the plain +aggregate enters MM_abs with weight ≈ 0, and the O(1) surplus coordinates +(healthy margins, large |σ|) dominate at every n. The adversary's only +lever is grinding the surplus weights down at fixed n (hence the +0.246 +and +0.064 floors at n = 48, and the rising transfers), or abandoning the +ladder geometry entirely (hence the +0.0101 free-support endpoint at +n = 8 — the genuinely open direction). + +**Why the window survives the ladder (so far).** 016's crossing is a +second-order-vs-first-order race: the ladder's aggregate is +a₁(n)λ + a₂(n)λ² + O(λ³) with a₁ ≥ 0 (Theorem C's perfect square) and +a₂ < 0, and the kill at λ ∈ [2, 2.5] lives where the quadratic term has +room to win. Restricting to λ ≤ 4.847/(n−3) freezes that race iff +a₁(n)·n stays bounded away from 0 — and on this genre it *grows* +(the replicated units contribute first-order squares additively while the +window shrinks). The joint climb's convergence to λ → 0 is the same fact +seen from the optimizer's side: inside the window there is no interior +minimum to exploit, only the trivial boundary zero. What this does NOT +rule out: a family engineered so the first-order square itself degenerates +(a₁ ~ 1/n² or faster) while a₂ < 0 — products are the equality set of a₁ +(014/015), so near-product families with negative curvature are the +designated hunting ground (lead 4). + +Reusable: + +- **The three-way disambiguation of 007 lead 2** — the margin-modulated + candidate is not one statement but three, and only the unsigned one is + alive. Any future proof effort must target MM_abs specifically, knowing + it lacks direct assembly meaning. +- **The exact MM certification path** (`exact_mm` in the engine): certified + interval evaluation of Plackett-root functionals over exact census + tables, with margin-pair caching that makes ladder-scale instances + affordable. First tool in the library that certifies a *nonlinear* + functional of the census (everything before was linear in log₂OR). +- **The product-measure structural zero** (MM_sec ≡ 0 at products, exact): + a free calibration anchor for any future engine touching these + quantities. +- The part-R friendly-genre battery: MM_sec sign map showing the violation + is specific to the two-block geometry. + +## Leads generated + +1. **Attack MM_abs where it is actually soft: free supports at n = 10–16 + seeded from the +0.0101 endpoint.** The tightest MM_abs margin came not + from any ladder but from a witness-seeded free climb at n = 8 + (`gap1c_partC.json` `best_free`, geometry saved). Re-run the anneal + with that endpoint as seed at n ∈ {10, 12, 16} with 2500+ steps and + coordinate-count mutations. Definite outcome: an in-regime MM_abs < 0 + witness (killing the last margin-modulated reading), or a floor that + survives deep search and licenses partial-proof effort (the ≤4-atom + and first-order-in-λ subclass theorems of 007 should port). +2. **Restate what the assembly actually needs, now that the secant form is + dead** (016 lead 5, sharpened). The chain-rule cost at a history IS the + secant quantity, and it is certified negative on the witness — so the + assembly cannot demand history-pointwise gain parity. Derive the + assembly's true requirement (presumably a coordinate-level or + λ-integrated inequality that tolerates per-history losses) and check it + on the witness and the θ\* ladder with this record's census. Deliverable + either way: a precise statement or a proof that no per-history + weighting can work (extend the part-R sign map into a no-go). +3. **Prove a₁(MU(n, r), θ) ≥ c/n^{1−ε} — first-order window-safety for the + replication genre.** a₁ is computable from 007's first-order machinery + (perfect-square coefficient, one term per history); on the ladder the + units contribute additively. A closed-form lower bound on the ladder's + a₁ would upgrade candidate II's survival from EVIDENCE to a theorem + *for this genre*; failure to prove it locates exactly which histories' + squares can vanish. Both outcomes are informative and the computation + is n-parametric, not a search. +4. **Hunt a₁-degenerate families: the near-product dip direction at window + λ.** Products are a₁'s equality set (014/015; the +6.4e-7 first-order + floor at n ≈ 48 is the smallest square seen). Build near-product + families along 014's slice-direction dip, scaled with n, and evaluate + A at λ_win(n): if A < 0 for some n, the window candidate dies exactly + where the plain control was weakest; if the dip's a₁ recovers (as it + did at fixed λ by n = 64), the window candidate survives its most + dangerous known direction. Cheap: reuse this record's engine unchanged. +5. **Skeptic pass on this record** (mandatory before ledger status): the + exact MM certificate is the first certified nonlinear census functional + in the library — re-derive `exact_mm` from 007 §1 + the Plackett + quadratic with an independent enclosure kit (017's interval-squaring + log₂ is structurally different from 013's atanh kit used here), re-run + the witness and n = 96 certificates, and re-check the secant identity + h₂(z̃) = h₂(z_OR(x, y)) symbolically rather than by round-trip. + +## References + +- This repo: `attempts/007-averaged-or-control.md` (lead 2 — the candidate + under test; §1 definitions; the witness), `attempts/013-*` (exact + standard, part C), `attempts/016-*` / `017-*` (ladder, θ\*, C2 window + correction, leads 1/3/4), `attempts/009-*` / `011-*` (λ-window law), + `attempts/008-*` / `012-*` (h(z_ρ) gain, assembly budgets). +- Tools/data: `explore/uc_gap1_candidates.py`; + `data/gap1c_part[A-F,R].json`, `data/gap1c_*.log`. +- No external papers consulted this cycle (record threshold 0.38271 per + prior records, Liu). diff --git a/problems/union-closed/attempts/019-skeptic-review-of-018.md b/problems/union-closed/attempts/019-skeptic-review-of-018.md new file mode 100644 index 0000000..2cac1c9 --- /dev/null +++ b/problems/union-closed/attempts/019-skeptic-review-of-018.md @@ -0,0 +1,336 @@ +# 019 — Skeptic review of 018 (surviving controls vs the ladder): adversarial verification + +- **Problem:** union-closed, `problems/union-closed/PROBLEM.md` +- **Date:** 2026-08-06 +- **Mode:** informed +- **Type:** adversarial verification of `018-surviving-controls-vs-ladder.md` + (default stance: refute). The decisive check is a **from-scratch exact + re-certification of the headline kill** with a kit sharing no code and no + algorithms with 018's path: own scaled-integer census written from 007 §1's + prose, Plackett-root enclosure by exact-rational Newton with an exact + sign-checked bracket (018 used blind dyadic bisection; my bracket endpoint + signs are proved symbolically and re-checked exactly per call), and a + dyadic interval-squaring log₂ enclosure (the idea of 017's kit, + re-implemented — NOT 013's atanh-series kit, which 018 imported). The + secant identity, the product-measure zero, and the degenerate-history + convention one-liner were re-derived by hand (below) and the last two also + engine-checked as exact identities. +- **Outcome in one line:** 018's headline is REAL — my independent + certificate `MM_sec ∈ [−0.022254658205655785176146, + −0.022254658205655785176145]` (width 5.9e-30) sits strictly inside 018's + enclosure and is negative, in-regime, dichotomy-exact — and no surviving + signed variant of the margin-modulated control was found in the fork 018 + left untested; but 018 ships one prose sentence its own checkpoint + contradicts (raw-ladder MM_sec/MM_der signs at n ≥ 96), one false + golden-ratio parenthetical repeated twice, a literal `PENDING-F` + placeholder for certification jobs that had not landed, and two smaller + imprecisions. VERIFIED_WITH_CORRECTIONS. +- **Tools:** `explore/uc_gap1c_skeptic.py` (parts S0–S4; stdlib only; + deterministic, fixed seeds; runtime ~15 s; checkpoints + `data/gap1csk_s[0-4].json`, log `data/gap1csk_run.log`). Command + reproducing every number below: + `python problems/union-closed/explore/uc_gap1c_skeptic.py 2>&1 | tee problems/union-closed/data/gap1csk_run.log` + Independence contract: no imports from `uc_gap1_candidates.py`, + `uc_or_avg.py`, `uc_or_avg_skeptic.py`, `uc_agg_ctrl_probe*.py`; witness + atoms, θ\* weights and search endpoints copied as DATA; the MU(n, r) + ladder and the dilution tuner re-implemented from 016 §1's prose and + checked atom-for-atom (`MU(7,1) ==` the 10 witness atoms, exact dict + equality). Two WebSearch queries for the novelty check (K5). +- **Sources:** 018 (record under attack, its engine read for claims only), + 007 §1 + lead 2 (the definitions 018 formalizes), 013 (the + exact-arithmetic standard), 016/017 (ladder spec, θ\*, window law), + gap1c checkpoints/logs (data). + +Notation as in 007/018: tilt `π ∝ u(A)u(B) t^{|A∩B|}`, `t = 2^λ`; at a +nondegenerate history the conditional 2×2 table has zero-margins +`x = P(A_i = 0)`, `y = P(B_i = 0)`, realized both-zero `z̃`, odds ratio OR; +`z_ρ(x, y)` the Plackett both-zero probability at odds ratio ρ; MM_sec / +MM_der / MM_abs as in 018's part B; record threshold 0.38271. + +## Claims attacked + +1. **(K1, headline)** The certified kill: MM_sec on 007's 10-atom witness at + exactly-rational t = 181/16 is negative + (`∈ [−0.02225465820566…, −0.02225465820566…]`), in-regime (all + elementwise marginals < 38271/100000), dichotomy exact — plus the three + supporting derivations: the secant identity `h₂(z̃) = h₂(z_OR(x, y))`, + `MM_sec ≡ 0` on product measures, and the degenerate-histories- + contribute-0 convention one-liner. +2. **(K2)** The definitional fork: that 007 lead 2 admits exactly the three + readings 018 tests, and that no natural *signed* variant survives — + attacked by constructing and testing the readings 018 ignored + (sensitivity evaluated at the **realized** OR instead of the target, + signed and unsigned; `|dh/dρ|` weighting). +3. **(K3)** The float survival numbers (EVIDENCE level, spot-checks): witness + MM_abs +2.354328 and MM_der −0.895542 at λ = 3.5; the θ\* ladder at + n = 96, λ = 2 (plain −0.000760, MM_sec −3.819e-4, MM_abs +0.969); the + part-D window value A = +3.747e-3 at n = 96, λ_win = 4.847/93, and the + a₁/a₂ fit arithmetic and `a₁·n` growth claims; the part-R perturbation + battery (re-run with different RNG seeds). +4. **(K4)** Scope honesty: Outcome claims vs what the logs/checkpoints show; + the free-climb +0.0101 endpoint's regime/nondegeneracy status; the + prior-art entry's leak_terms/gaps. +5. **(K5)** Novelty: is the margin-modulated statement (and its kill) + plausibly a rediscovery? + +## Refutations found + +None headline-level: the certified kill, both survival claims, and the fork +analysis all stand. Four corrections and one cosmetic: + +### C1. The raw-ladder MM_sec/MM_der sign sentence is contradicted by 018's own checkpoint +Location: 018 part B, "On the ladders at λ = 2 (raw witness weights and +016's θ\*), MM_sec is negative at every n ∈ {16, …, 160} (…), MM_der +negative throughout". + +False for the **raw** ladder at n ∈ {96, 128, 160}: 018's own +`gap1c_partB.json` has MM_sec **+6.72e-3 / +6.05e-3 / +5.25e-3** and MM_der +**+0.99 / +0.97 / +0.95** there (plain +1.16 / +0.99 / +0.83 — the +`tune_dil` bisection lands on a different dilution branch at n ≥ 96, mm +0.311 vs 0.359 at n = 64, and the whole geometry flips sign). Reproduced +independently: my spec-rebuilt raw ladder at n = 96 gives MM_sec ++6.719528e-3, MM_der +0.9856 (dil 251.49), against −3.400008e-3 at n = 64 +(part S3 — both matching 018's checkpoint to all printed digits, so the +error is in the prose, not the data). **Corrected statement:** at λ = 2, +MM_sec and MM_der are negative on the θ\* ladder at every n ∈ {16..160} and +on the raw ladder for n ≤ 64; on the raw ladder's n ≥ 96 dilution branch +both are positive. No downstream damage — the *survival* claims only need +MM_abs > 0 (true on every row), and the *kill* is carried by the witness — +but the sentence as written overstates how universally the signed forms +fail on ladders. + +### C2. The golden-ratio parenthetical is false (twice) +Location: 018 Approach, "(z_1(x, x) = x² crosses 1/2 exactly at the +(3−√5)/2 barrier — the sensitivity sign boundary is the record threshold's +golden-ratio structure again)"; repeated in Why-it-failed as "The +golden-ratio coincidence (z crosses 1/2 exactly at marginal (3−√5)/2 for +equal margins) means this sign flip is intrinsic to the record regime". + +Arithmetic: at element marginal p = (3−√5)/2 = 0.381966, the zero-margin is +x = 1 − p = (√5−1)/2 = 0.618034 and `x² = 0.381966 = p` — the φ identity +`φ² = 1 − φ` makes x² equal **the marginal itself**, not 1/2. The actual +z = 1/2 crossing at ρ = 1 is at x = 2^{−1/2}, i.e. element marginal +1 − 2^{−1/2} ≈ **0.292893**; at ρ = 2^{3.5} it moves to marginal ≈ 0.389113 +(λ-dependent; its proximity to 0.38271 at λ = 3.5 is a coincidence of that +λ and is presumably where the confusion came from). Part S4. What survives: +the load-bearing claim that in-regime histories straddle z = 1/2 is true +(marginals < 0.38271 force x > 0.617, so z̃ ranges over both sides of 1/2), +and the witness's surplus rows do sit at z̃ > 1/2 — the *mechanism* stands, +the golden-ratio numerology attached to it is wrong, exactly the genre of +flourish 004 already killed once in 003 (R2's "p_h ≈ φ"). + +### C3. Part F shipped with a `PENDING-F` placeholder; only 1 of 4 certification jobs had landed +At review time 018's `--cert` pass had completed only job (1) (the witness +MM cert — the one the Outcome claims); jobs (2)–(4) (MM on the θ\* kill +instances at n = 96/160 at t = 4, window points at t = 51/50) were still +running, `gap1c_partF.json` had one row, `gap1c_cert.log` ends at the +witness line, and the record's §F contains a literal "`- PENDING-F`" +bullet. The Outcome section is careful to claim only the witness +certificate, so nothing certified is overstated — but a record should not +ship an unresolved placeholder, and any future reader should treat the +n = 96/160 MM certificates and window certificates as **absent from 018** +unless a follow-up lands them. (Float values at those points are covered by +K3 below.) + +### C4. "positive only for λ ≲ 0.7" — the witness MM_sec crossing is at λ ≈ 0.83 +018 part B. My profile: MM_sec = +2.00e-4 at λ = 0.75, +4.95e-5 at 0.8, +−3.21e-4 at 0.9 (part S2), so the sign change is between 0.8 and 0.9, not +at ≈ 0.7. Consistent with 018's own part-R grid (+ at 0.5, − at 1.0), which +is too coarse to support the "0.7". Minor: the claim's role (positivity in +the first-order-dominated region) is unaffected. + +### C5 (cosmetic). Runtime drift +Tools bullet says parts A–D "~35 min"; the engine docstring says 20–40 min; +`gap1c_run.log` shows 3496 s ≈ 58 min. Recorded per 013 R1 precedent. + +## Claims that survive (and what was done to break them) + +### 1. The headline kill (K1) — re-certified from scratch; enclosures agree at width 5.9e-30 + +- **Hand re-derivations.** (i) *Secant identity:* on the coupling range + `z ∈ (max(0, x+y−1), min(x, y))`, + `d/dz ln ρ(z) = 1/z + 1/(1−x−y+z) + 1/(x−z) + 1/(y−z) > 0` (all four + terms positive), so the margins-plus-odds-ratio parametrization of 2×2 + tables is bijective: the realized table's z̃ IS `z_OR(x, y)`, hence + `h₂(z̃) = h₂(z_OR(x, y))` and, by the mean value theorem in + λ' = log₂ ρ, `h₂(z̃) − h₂(z_t) = σ(λ̄)·(log₂OR − λ)` for some λ̄ on the + secant. 018's "assembly-exact secant form" gloss is correct. (ii) + *Product zero:* a product potential factorizes the kernel + `Π_j u_j(a_j)u_j(b_j)2^{λ a_j b_j}`; conditioning on any history leaves + the response factor `∝ [[u₀u₀, u₀u₁],[u₁u₀, u₁u₁2^λ]]`, so OR = 2^λ at + every nondegenerate history (and Sinkhorn uniqueness carries this to + product μ); by (i), z̃ = z_t row by row, so MM_sec ≡ 0 **term by term**, + not just in aggregate. Engine: exact product anchor at w = 3/7, n = 5, + t = 4 — plain aggregate exactly 0 (rational identity), MM_sec enclosure + contains 0 at width 9.3e-30 (part S1). (iii) *Convention one-liner:* a + margin-degenerate history has x ∈ {0, 1} or y ∈ {0, 1}; then z̃ is pinned + (x = 0 or y = 0 ⟹ z̃ = 0; x = 1 ⟹ z̃ = y; y = 1 ⟹ z̃ = x) and + `z_ρ` is pinned to the same boundary value for **every** ρ > 0, so the + secant contribution is exactly 0 — including degenerate histories changes + only the denominator. Checked as an exact identity on every degenerate + positive-mass history of the witness (flag `deg_zero_ok`, part S1). +- **The certificate.** Own kernel (scaled integers), own census, own root + and log₂ enclosures (part S1): + + witness, t = 181/16: + MM_sec ∈ [−0.022254658205655785176146, −0.022254658205655785176145] + plain aggregate ∈ [+1.844669005340829561098944, …945] + max elementwise marginal = 0.3177701854 (exact < 38271/100000) + dichotomy exact; degenerate rows contribute 0 exactly; 12 rows + + Strictly inside 018's 21-digit enclosure and matching 013's aggregate + certificate digit-for-digit — three structurally different kits now agree + on the same rational-input theorem. The **2-digit tidy witness** is also + certified negative here (−0.0222736748, max marginal 0.3130, in-regime), + upgrading 018's float −0.0223 to a certificate. +- **Kit validity attacked first** (part S0): log₂ enclosure vs 67 known / + random values plus additivity (max width 9.1e-30); Plackett Newton root + vs the float solver and the closed-form round-trip ρ(z_root)/t = 1 to + 1e-9 on 60 random (x, y, t), bracket endpoint sign identities + `F(xy) = (1−t)xy(1−x)(1−y)`, `F(min) = min·(1−max)` asserted exactly per + call; h₂ enclosure spot-checks; ladder-spec equality `MU(7,1) ==` + witness. All pass. + +**CONFIRMED, and strengthened (tidy witness now certified).** The +margin-modulated control in its secant reading is dead at n = 7 in-regime; +by the secant identity + MVT the *derivative-at-target* reading cannot be +salvaged as "the" chain-rule reading either (float MM_der −0.8955 +reproduced; not separately certified here, matching 018's own scoping). + +### 2. The definitional fork (K2) — fair, and the untested variants change nothing + +007 lead 2's text ("weighted by the sensitivity of h(z_ρ(x̃,ỹ)) to ρ … +conjecture E[σ(x̃,ỹ)·(log₂OR − λ)] ≥ 0 … if the weighted mean is ≥ λ") +underdetermines where σ is evaluated; 018's three readings cover the target +and secant evaluations, and its "weighted mean" gloss honestly favors the +unsigned reading. The natural fourth family 018 skipped — σ evaluated at +the **realized** OR — was built and tested here (part S2): + + witness (λ = 3.5): MM_der_rlz −1.458 MM_abs_rlz +2.133 MM_absrho_rlz +1.808 + θ* ladder n = 96, λ = 2: MM_der_rlz −4.35e-1 MM_abs_rlz +9.34e-1 MM_absrho_rlz +8.80e-1 + +The signed realized-OR reading dies on the witness even harder than +MM_der; both unsigned realized variants (including the |dh/dρ| weighting, +which at the realized OR is a genuinely different weight, not a constant +multiple) behave like MM_abs — positive on the witness and on the kill +instance. **No surviving signed variant found; the signed/unsigned +dichotomy 018 draws is robust to the readings it did not test.** (The two +extra unsigned survivors inherit MM_abs's defect: not the derivative of any +gain.) + +### 3. The float survival numbers (K3) — reproduced on a spec-rebuilt ladder, different seeds + +- Witness: MM_abs **+2.354328**, MM_der **−0.895542**, MM_sec −0.022255 — + all match (part S3). +- θ\* ladder, n = 96, λ = 2, my own `tune_dil` (dil 31.6228): plain + **−0.000760** (−7.598652e-4), MM_sec **−3.819411e-4**, MM_abs + **+0.969045**, mm 0.309 — all match 018 to printed precision, on a + builder written from 016 §1's prose, not imported. +- Window point n = 96, λ_win = 4.847/93 = 0.052118: A = **+3.747072e-3** ✓. + The a₁/a₂ 2×2 solves re-derive for **all 10** stored part-D rows + (rel. 1e-9), and the `a₁·n` growth sequence on θ\* re-computes as + 4.73 → 7.19 → 10.37 → 13.50 → 18.13 ✓ (raw rows re-derive too; note the + raw fit's a₂ changes sign across n — +0.027 at n = 160 — so 018's "same + shape" for raw is loose, though its a₁·n ≈ 66 claim is right). +- Perturbation battery with different seeds (7 and 424242, 20 draws each, + 3% multiplicative): **0/40 failures**, MM_sec ∈ [−0.0230, −0.0216] — + 018's seed-2024 result is not a seed artifact. +- The free-climb endpoint (from `gap1c_partC.json` `best_free`): + re-evaluated with my census — MM_abs **+1.007614e-2**, MM_sec + −8.327e-5, max marginal **0.153275 < 0.38271** (genuinely in-regime), + dichotomy ok, H = 1.019 bits (nondegenerate, 14 atoms). The tightest + MM_abs margin is real (part S4). + +CONFIRMED (as EVIDENCE, at 018's own scope). + +### 4. Scope honesty (K4) — Outcome matches the logs, with the C1/C3 caveats + +"Positive on everything tried" for MM_abs: true of every row in +`gap1c_part[B,C,R].json` (checked). The window-candidate claims match +`gap1c_partD.json` including the λ → 0 boundary convergence of the joint +climb (+1.66e-5 at λ = 0.0008). Part E's dilution-branch jump at +n = 288 (dil 80 → 160) is as recorded. One scope note short of a +correction: of the 8 free-support climbs, only the three witness-seeded +endpoints are in-regime (the five random-seeded endpoints have max +marginal 0.384–0.663), so the "8 climbs" battery carries less in-regime +weight than its count suggests — the floor claim itself is in-regime and +correct. The prior-art entry's leak_terms/gaps do name the new findings +(MM_sec/MM_der/MM_abs, secant form, abs-sensitivity-or-control, +0.0222546582); status REFUTED with the two survivors relegated to `gaps` +is the right bookkeeping. + +### 5. Novelty (K5) — checked, low risk + +Two web searches ran (2026-08-06): "union-closed sets conjecture Plackett +odds ratio entropy coupling sensitivity weighted" and a narrower +odds-ratio/tilt/secant query. Results surface the known entropy/coupling +literature (Gilmer arXiv:2211.09055 line, Sawin, Chase–Lovett, Liu +arXiv:2306.08824, Yu–Cambie dimension-free) — nothing resembling a +sensitivity-weighted / margin-modulated conditional-OR control or its +refutation. The statement is internal to this lab's 007-lead-2 route, as +expected; the kill is a result about the lab's own conjecture, so novelty +risk is essentially nil. + +## Verdict + +| # | Claim | Verdict | +|---|-------|---------| +| 1 | Certified witness kill of MM_sec at t = 181/16, in-regime, dichotomy exact | **CONFIRMED** — independent certificate, width 5.9e-30, strictly inside 018's enclosure; tidy witness additionally certified (−0.02227) | +| 1b | Secant identity; product-measure zero; degenerate-contribution-0 | **CONFIRMED** — re-derived by hand; product zero and degeneracy checked as exact identities | +| 2 | Three-way fork; only unsigned readings survive | **CONFIRMED and extended** — realized-OR variants tested: signed dies (−1.458 on witness), unsigned survives; no surviving signed variant exists in the natural family | +| 3a | Witness/ladder float numbers; window value; fit arithmetic; a₁·n growth | **CONFIRMED** (spec-rebuilt ladder, own tuner; all 10 fit rows re-derive) | +| 3b | Ladder sign prose at λ = 2 | **CORRECTED (C1)** — raw ladder at n ≥ 96 has MM_sec, MM_der > 0 per 018's own checkpoint; θ\*-only statement survives | +| 3c | Perturbation robustness of the kill | **CONFIRMED** with different seeds (0/40 failures) | +| 4 | Outcome scope; free-climb endpoint in-regime; prior-art bookkeeping | **CONFIRMED** (with the 3-of-8-in-regime scope note; C3 for the PENDING-F placeholder) | +| 5 | Golden-ratio sign-boundary parenthetical | **REFUTED (C2)** — x² = marginal, not 1/2, at the barrier; true ρ=1 crossing at marginal 1 − 2^{−1/2} ≈ 0.293; straddling claim itself survives | + +**Net assessment:** 018's headline — the margin-modulated OR control is +REFUTED in both signed readings at n = 7, certified float-free on 007's own +witness — holds and is now double-certified by structurally disjoint kits; +its two survival claims stand at their stated EVIDENCE scope, and the fork +analysis is complete once the realized-OR variants are added (they change +nothing). The corrections are all prose-level: a checkpoint-contradicting +sentence (C1), a numerological flourish (C2, the same failure genre 004 +flagged in 003), an unlanded-certification placeholder (C3), and two +imprecisions (C4, C5). Ledger status VERIFIED_WITH_CORRECTIONS; +`abs-sensitivity-or-control` and `lambda-restricted-or-control` are the +correct surviving gap tags. + +## Residual risk + +- **Shared-definition risk** (as in 013): every engine including this one + formalizes the census from 007 §1's prose. Exact arithmetic removes + numerical error, not a conceptual error in that shared definition; the + +1.844669005 aggregate anchor ties this review to 013's independently + hand-derived census, which is the main mitigation. +- **Part-F jobs (2)–(4)** of 018 were still running at review close; + whatever they produce is unreviewed. The MM_der reading remains float + EVIDENCE (neither 018 nor this review certified it; the secant + certificate plus the MVT tie is the argument that it cannot be the + assembly's reading anyway). +- **Search-completeness claims** (θ-opt floors, free climbs, window climbs) + were spot-checked for reproducibility, not re-run at scale; they remain + EVIDENCE with 018's own "one adversary genre" caveat. My variant zoo + covered the natural sensitivity-evaluation points (target, secant, + realized; λ- vs ρ-derivative) but not arbitrary nonnegative weight + functions of the margins — a survivor family there is unconstrained by + this review (and by design would drift further from chain-rule meaning). +- The two unsigned realized-OR variants found positive here were tested at + two instances only; they are observations, not survival claims. + +## References + +- Reviewed record: `problems/union-closed/attempts/018-surviving-controls-vs-ladder.md`; + its engine `explore/uc_gap1_candidates.py`; checkpoints + `data/gap1c_part[A-F,R].json`, logs `data/gap1c_{run,ext,robust,cert}.log`. +- Context: `attempts/007-averaged-or-control.md` (§1 census, lead 2, part-T + witness), `attempts/013-skeptic-review-of-007.md` (exact standard, + +1.844669005 anchor), `attempts/016-*` / `017-*` (MU(n, r) spec, θ\*, + window law, interval-squaring idea), `attempts/004-*` (review shape; + φ-numerology precedent). +- This review's tool/data: `explore/uc_gap1c_skeptic.py`; + `data/gap1csk_s[0-4].json`; `data/gap1csk_run.log`. +- Novelty check (WebSearch, 2026-08-06): arXiv:2211.09055 (Gilmer), + arXiv:2306.08824 (Liu), arXiv:2212.12500 (Yu), arXiv:2305.19338, + arXiv:2306.12351 (survey) — no margin-modulated/sensitivity-weighted OR + control in the literature surfaced. diff --git a/problems/union-closed/data/gap1c_cert.log b/problems/union-closed/data/gap1c_cert.log new file mode 100644 index 0000000..0554166 --- /dev/null +++ b/problems/union-closed/data/gap1c_cert.log @@ -0,0 +1,12 @@ +[ 0s] [A] witness plain aggregate (lam 3.5): +1.844754 (013 certified +1.844669) +[ 0s] [A] census_mm plain-agg cross-check: +1.844754 (diff 2.22e-16) +[ 2s] [A] theta* ladder n=96 lam=2 plain aggregate: -0.000759865 (016 certified -0.000759865) +[ 2s] [A] Plackett round-trip on 12 witness rows: max |z_rho(x,y,OR) - z~| = 1.55e-15 +[ 2s] [A] secant vs derivative (dl=1e-3): 0.000027704 vs 0.000027709 (rel diff 2.0e-04) +[ 2s] [A] MM padding invariance MU(20,1) vs witness: diff 0.00e+00 +[ 2s] [A] anchors ALL PASS +[ 7s] [F] exact MM mm_witness_t181/16: MM_sec in [-2.225465821e-02, -2.225465821e-02] CERTIFIED NEGATIVE (KILL); plain in [+1.844669, +1.844669]; in-regime True; dich True; rows 12 (12 distinct margins); 4s +[ 1443s] [F] exact MM mm_theta*_n96_t4: MM_sec in [-3.819411484e-04, -3.819411484e-04] CERTIFIED NEGATIVE (KILL); plain in [-0.000760, -0.000760]; in-regime True; dich True; rows 368 (368 distinct margins); 1436s +[ 5513s] [F] exact MM mm_theta*_n160_t4: MM_sec in [-2.229998913e-04, -2.229998913e-04] CERTIFIED NEGATIVE (KILL); plain in [-0.024226, -0.024226]; in-regime True; dich True; rows 624 (624 distinct margins); 4069s +[ 5653s] [F] exact plain window_theta*_n96_t51/50: agg in [+2.092188747e-03, +2.092188747e-03] CERTIFIED POSITIVE; in-regime True; dich True; 140s +[ 6120s] [F] exact plain window_theta*_n160_t51/50: agg in [+1.802171374e-03, +1.802171374e-03] CERTIFIED POSITIVE; in-regime True; dich True; 467s diff --git a/problems/union-closed/data/gap1c_ext.log b/problems/union-closed/data/gap1c_ext.log new file mode 100644 index 0000000..eb02bcf --- /dev/null +++ b/problems/union-closed/data/gap1c_ext.log @@ -0,0 +1,6 @@ +[ 54s] [E] raw ladder n=160: honest min agg +0.190168 (dil 80.0, mm 0.261) +[ 154s] [E] raw ladder n=192: honest min agg +0.150420 (dil 80.0, mm 0.296) +[ 358s] [E] raw ladder n=224: honest min agg +0.123176 (dil 80.0, mm 0.330) +[ 699s] [E] raw ladder n=256: honest min agg +0.103745 (dil 80.0, mm 0.362) +[ 1125s] [E] raw ladder n=288: honest min agg +0.213646 (dil 160.0, mm 0.259) +[ 1749s] [E] raw ladder n=320: honest min agg +0.180537 (dil 160.0, mm 0.252) diff --git a/problems/union-closed/data/gap1c_partA.json b/problems/union-closed/data/gap1c_partA.json new file mode 100644 index 0000000..9465931 --- /dev/null +++ b/problems/union-closed/data/gap1c_partA.json @@ -0,0 +1,14 @@ +{ + "all_ok": true, + "kill_repro_agg": -0.000759865185986192, + "ok_cross": true, + "ok_kill_repro": true, + "ok_padding": true, + "ok_roundtrip": true, + "ok_secant_deriv": true, + "ok_witness": true, + "witness_agg": 1.8447543150919639, + "witness_mm_abs": 2.3543277749368867, + "witness_mm_der": -0.8955419815884347, + "witness_mm_sec": -0.022255048835182943 +} diff --git a/problems/union-closed/data/gap1c_partB.json b/problems/union-closed/data/gap1c_partB.json new file mode 100644 index 0000000..32a326a --- /dev/null +++ b/problems/union-closed/data/gap1c_partB.json @@ -0,0 +1,274 @@ +{ + "rows": [ + { + "agg_plain": 1.844754315091964, + "dichotomy_ok": true, + "lambda": 3.5, + "max_marginal": 0.3177685743531096, + "mm_abs": 2.3543277749368867, + "mm_der": -0.8955419815884347, + "mm_sec": -0.022255048835182943, + "n": 7, + "name": "witness" + }, + { + "agg_plain": 0.633032913009122, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.24171985207861715, + "mm_abs": 1.2399647249043146, + "mm_der": -0.23901131572931644, + "mm_sec": -0.005702361089221228, + "n": 16, + "name": "raw_ladder", + "per_i_sec_argmin": 1, + "per_i_sec_min": -0.030980215953081758 + }, + { + "agg_plain": 0.3530399949635046, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.28266059516623426, + "mm_abs": 1.1569587897360674, + "mm_der": -0.3915035870918062, + "mm_sec": -0.004382961258397616, + "n": 32, + "name": "raw_ladder", + "per_i_sec_argmin": 1, + "per_i_sec_min": -0.029480949175043965 + }, + { + "agg_plain": 0.23979434939727393, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.3218587336721369, + "mm_abs": 1.0856072080402273, + "mm_der": -0.5373214456813569, + "mm_sec": -0.0037829688962846245, + "n": 48, + "name": "raw_ladder", + "per_i_sec_argmin": 1, + "per_i_sec_min": -0.02823105801081694 + }, + { + "agg_plain": 0.180142390793786, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.359027940735784, + "mm_abs": 1.0711192931328388, + "mm_der": -0.6193039068293124, + "mm_sec": -0.0034000080267896074, + "n": 64, + "name": "raw_ladder", + "per_i_sec_argmin": 1, + "per_i_sec_min": -0.027177197503036977 + }, + { + "agg_plain": 1.1630136355382952, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.3110651388915359, + "mm_abs": 1.5146875468214835, + "mm_der": 0.9856109084724816, + "mm_sec": 0.0067195278438671215, + "n": 96, + "name": "raw_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.010555308511683173 + }, + { + "agg_plain": 0.9862096993096527, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.3074572445883211, + "mm_abs": 1.503923356736136, + "mm_der": 0.9701218099494555, + "mm_sec": 0.006054829973077529, + "n": 128, + "name": "raw_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011037618585548132 + }, + { + "agg_plain": 0.8281642503273758, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.30373848323979635, + "mm_abs": 1.495748008164421, + "mm_der": 0.9523977388874132, + "mm_sec": 0.005245484584956438, + "n": 160, + "name": "raw_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011540314170745688 + }, + { + "agg_plain": 0.2969899929628336, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.3105098092684316, + "mm_abs": 0.9641138874004778, + "mm_der": -0.1766320434880563, + "mm_sec": -0.0024155578313648047, + "n": 16, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.01139100001675577 + }, + { + "agg_plain": 0.11682621325497722, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.31014142224499625, + "mm_abs": 0.9650982824301354, + "mm_der": -0.17558946988559274, + "mm_sec": -0.0011842271231937503, + "n": 32, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011370702155838991 + }, + { + "agg_plain": 0.05783176247572485, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.3097728879199251, + "mm_abs": 0.9660836735527687, + "mm_der": -0.17451471702065977, + "mm_sec": -0.0007813111666197384, + "n": 48, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011350428069592581 + }, + { + "agg_plain": 0.02850273099649828, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.30940421458833184, + "mm_abs": 0.9670699695240861, + "mm_der": -0.17340728494617533, + "mm_sec": -0.0005812056949078478, + "n": 64, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011330177988461355 + }, + { + "agg_plain": -0.0007598651859861864, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.308666483724311, + "mm_abs": 0.9690449182509442, + "mm_der": -0.17109234495587256, + "mm_sec": -0.0003819411483889216, + "n": 96, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011289750747750755 + }, + { + "agg_plain": -0.015405727482615319, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.3079282945712954, + "mm_abs": 0.9710224269856843, + "mm_der": -0.1686404928198903, + "mm_sec": -0.0002825734878772398, + "n": 128, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011249422215504623 + }, + { + "agg_plain": -0.024225909314808017, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.30718971090418795, + "mm_abs": 0.9730018155035021, + "mm_der": -0.166047436000507, + "mm_sec": -0.00022299987185031748, + "n": 160, + "name": "theta*_ladder", + "per_i_sec_argmin": 2, + "per_i_sec_min": -0.011209194124741879 + }, + { + "agg_plain": 0.02391225083971442, + "dichotomy_ok": true, + "lambda": 0.5, + "max_marginal": 0.27710387955748017, + "mm_abs": 0.19879853101974007, + "mm_der": -0.01364784750258565, + "mm_sec": -3.660908777634866e-05, + "n": 96, + "name": "theta*_lamprof" + }, + { + "agg_plain": 0.02520669097823305, + "dichotomy_ok": true, + "lambda": 1.0, + "max_marginal": 0.2877182386354047, + "mm_abs": 0.41891727286318536, + "mm_der": -0.05398782289124554, + "mm_sec": -0.00014430520460107475, + "n": 96, + "name": "theta*_lamprof" + }, + { + "agg_plain": 0.01316272905887752, + "dichotomy_ok": true, + "lambda": 1.5, + "max_marginal": 0.29871417296032526, + "mm_abs": 0.6606551833151457, + "mm_der": -0.1117600580419466, + "mm_sec": -0.00027987223188268663, + "n": 96, + "name": "theta*_lamprof" + }, + { + "agg_plain": -0.0007598651859861864, + "dichotomy_ok": true, + "lambda": 2.0, + "max_marginal": 0.308666483724311, + "mm_abs": 0.9690449182509442, + "mm_der": -0.17109234495587256, + "mm_sec": -0.0003819411483889216, + "n": 96, + "name": "theta*_lamprof" + }, + { + "agg_plain": -0.005632293436193985, + "dichotomy_ok": true, + "lambda": 2.5, + "max_marginal": 0.3654293979953328, + "mm_abs": 1.3811576706882305, + "mm_der": -0.21528866275900307, + "mm_sec": -0.0004120991121538551, + "n": 96, + "name": "theta*_lamprof" + }, + { + "agg_plain": 0.9531187848700836, + "dichotomy_ok": true, + "lambda": 3.0, + "max_marginal": 0.30441202087952673, + "mm_abs": 2.3628479508755267, + "mm_der": -0.2764911887152964, + "mm_sec": -0.005992614594638558, + "n": 96, + "name": "theta*_lamprof" + }, + { + "agg_plain": 1.0164331219393128, + "dichotomy_ok": true, + "lambda": 3.5, + "max_marginal": 0.30378680812688097, + "mm_abs": 2.9152414589358813, + "mm_der": -0.8121616087475895, + "mm_sec": -0.00805133063180297, + "n": 96, + "name": "theta*_lamprof" + } + ] +} diff --git a/problems/union-closed/data/gap1c_partC.json b/problems/union-closed/data/gap1c_partC.json new file mode 100644 index 0000000..183fc9f --- /dev/null +++ b/problems/union-closed/data/gap1c_partC.json @@ -0,0 +1,249 @@ +{ + "best_free": { + "lambda": 2.0, + "max_marginal": 0.15327507789699263, + "mm_abs": 0.01007613581320645, + "mm_sec": -8.327292197854983e-05, + "trial": 1, + "u": { + "00000000": 347.00624724906413, + "00000010": 0.3856345187269309, + "00000100": 49.26776497822746, + "00001000": 5.880550223622179, + "00100000": 7.3122511471755605, + "00100001": 0.7689077882839955, + "00110000": 0.2251282437730199, + "01000000": 7.2335817956389175, + "01000001": 0.2862201990027687, + "01001100": 0.20696646980565703, + "01010001": 0.007352482336842667, + "01110001": 0.00011105499264138503, + "10000001": 0.04144629090634847, + "11010010": 0.010635964275412581 + } + }, + "best_theta_by_lam": { + "2.0": { + "A1": 14.026917090492047, + "A2": 39.20299927745608, + "B0": 214.01979504223078, + "B1": 23.385373708349576, + "B2": 25.081828502933508, + "dil": 24.658527497276282, + "light": 0.3159341281131832, + "resp": 0.10333667754200809 + }, + "3.5": { + "A1": 11.449048690253502, + "A2": 2.225348514743002, + "B0": 3981.307793321206, + "B1": 115.72886820155412, + "B2": 28.87381380755789, + "dil": 145.62480793043528, + "light": 0.021726810714908097, + "resp": 0.14038034413695086 + } + }, + "rows": [ + { + "agg_plain": 0.46911645259040685, + "lambda": 2.0, + "max_marginal": 0.36256946343575214, + "mm_abs": 0.2462548461106482, + "mm_der": -0.2462548461106482, + "mm_sec": -0.0005344705313283675, + "n": 48, + "stage": "opt", + "theta": { + "A1": 14.026917090492047, + "A2": 39.20299927745608, + "B0": 214.01979504223078, + "B1": 23.385373708349576, + "B2": 25.081828502933508, + "dil": 24.658527497276282, + "light": 0.3159341281131832, + "resp": 0.10333667754200809 + } + }, + { + "agg_plain": 0.4487391251769076, + "lambda": 2.0, + "max_marginal": 0.36399543884230884, + "mm_abs": 0.23871013319799322, + "mm_der": -0.21485757935001829, + "mm_sec": -0.0004267747237145901, + "n": 64, + "stage": "transfer", + "theta": { + "A1": 14.026917090492047, + "A2": 39.20299927745608, + "B0": 214.01979504223078, + "B1": 23.385373708349576, + "B2": 25.081828502933508, + "dil": 31.622776601683793, + "light": 0.3159341281131832, + "resp": 0.10333667754200809 + } + }, + { + "agg_plain": 0.5817590696905286, + "lambda": 2.0, + "max_marginal": 0.2474927952429104, + "mm_abs": 1.003747817027549, + "mm_der": -0.8063980555875502, + "mm_sec": -0.005378083218066918, + "n": 96, + "stage": "transfer", + "theta": { + "A1": 14.026917090492047, + "A2": 39.20299927745608, + "B0": 214.01979504223078, + "B1": 23.385373708349576, + "B2": 25.081828502933508, + "dil": 251.48668593658707, + "light": 0.3159341281131832, + "resp": 0.10333667754200809 + } + }, + { + "agg_plain": 0.505694093327669, + "lambda": 2.0, + "max_marginal": 0.2421794141835934, + "mm_abs": 1.0554451323512237, + "mm_der": -0.8672663711779388, + "mm_sec": -0.004350514709457147, + "n": 128, + "stage": "transfer", + "theta": { + "A1": 14.026917090492047, + "A2": 39.20299927745608, + "B0": 214.01979504223078, + "B1": 23.385373708349576, + "B2": 25.081828502933508, + "dil": 251.48668593658707, + "light": 0.3159341281131832, + "resp": 0.10333667754200809 + } + }, + { + "agg_plain": 0.445455317398738, + "lambda": 2.0, + "max_marginal": 0.23674751535821315, + "mm_abs": 1.0940518759999978, + "mm_der": -0.9124125002227456, + "mm_sec": -0.003574521573993709, + "n": 160, + "stage": "transfer", + "theta": { + "A1": 14.026917090492047, + "A2": 39.20299927745608, + "B0": 214.01979504223078, + "B1": 23.385373708349576, + "B2": 25.081828502933508, + "dil": 251.48668593658707, + "light": 0.3159341281131832, + "resp": 0.10333667754200809 + } + }, + { + "agg_plain": 2.455956440780101, + "lambda": 3.5, + "max_marginal": 0.040513655112411485, + "mm_abs": 0.06393336087664175, + "mm_der": -0.060425397346137506, + "mm_sec": -0.0001513088871274416, + "n": 48, + "stage": "opt", + "theta": { + "A1": 11.449048690253502, + "A2": 2.225348514743002, + "B0": 3981.307793321206, + "B1": 115.72886820155412, + "B2": 28.87381380755789, + "dil": 145.62480793043528, + "light": 0.021726810714908097, + "resp": 0.14038034413695086 + } + }, + { + "agg_plain": 2.5293555755068486, + "lambda": 3.5, + "max_marginal": 0.04266368513402975, + "mm_abs": 0.1410008210898418, + "mm_der": -0.12943460040833932, + "mm_sec": -8.540340344697633e-05, + "n": 64, + "stage": "transfer", + "theta": { + "A1": 11.449048690253502, + "A2": 2.225348514743002, + "B0": 3981.307793321206, + "B1": 115.72886820155412, + "B2": 28.87381380755789, + "dil": 31.622776601683793, + "light": 0.021726810714908097, + "resp": 0.14038034413695086 + } + }, + { + "agg_plain": 2.5774967074398254, + "lambda": 3.5, + "max_marginal": 0.04431310983729134, + "mm_abs": 0.14640612639413778, + "mm_der": -0.13489530938480415, + "mm_sec": -5.831230466766774e-05, + "n": 96, + "stage": "transfer", + "theta": { + "A1": 11.449048690253502, + "A2": 2.225348514743002, + "B0": 3981.307793321206, + "B1": 115.72886820155412, + "B2": 28.87381380755789, + "dil": 31.622776601683793, + "light": 0.021726810714908097, + "resp": 0.14038034413695086 + } + }, + { + "agg_plain": 2.5952841138477076, + "lambda": 3.5, + "max_marginal": 0.04597580044317281, + "mm_abs": 0.15241395905947425, + "mm_der": -0.1409603097412247, + "mm_sec": -4.511368763805838e-05, + "n": 128, + "stage": "transfer", + "theta": { + "A1": 11.449048690253502, + "A2": 2.225348514743002, + "B0": 3981.307793321206, + "B1": 115.72886820155412, + "B2": 28.87381380755789, + "dil": 31.622776601683793, + "light": 0.021726810714908097, + "resp": 0.14038034413695086 + } + }, + { + "agg_plain": 2.601084482979765, + "lambda": 3.5, + "max_marginal": 0.04765146971423037, + "mm_abs": 0.1590279579760817, + "mm_der": -0.1476332080558259, + "mm_sec": -3.741645142582835e-05, + "n": 160, + "stage": "transfer", + "theta": { + "A1": 11.449048690253502, + "A2": 2.225348514743002, + "B0": 3981.307793321206, + "B1": 115.72886820155412, + "B2": 28.87381380755789, + "dil": 31.622776601683793, + "light": 0.021726810714908097, + "resp": 0.14038034413695086 + } + } + ] +} diff --git a/problems/union-closed/data/gap1c_partD.json b/problems/union-closed/data/gap1c_partD.json new file mode 100644 index 0000000..3193e5e --- /dev/null +++ b/problems/union-closed/data/gap1c_partD.json @@ -0,0 +1,478 @@ +{ + "rows": [ + { + "a1": 0.09858947695317884, + "a2": -0.048956262221548634, + "fit_check_at_lw": { + "actual": 0.010052663795943606, + "in_fit": false, + "pred": 0.010051207048968154 + }, + "lamwin": 0.10771111111111112, + "n": 48, + "name": "theta*_window", + "points": [ + { + "agg": 0.010052663795943606, + "lambda": 0.10771111111111112, + "max_marginal": 0.2698474542337888, + "per_i_min": 0.004619549413992272 + }, + { + "agg": 0.005167597288864582, + "lambda": 0.05385555555555556, + "max_marginal": 0.26888999277663367, + "per_i_min": 0.0024943718123722966 + }, + { + "agg": 0.002619297085527417, + "lambda": 0.02692777777777778, + "max_marginal": 0.26841858648074546, + "per_i_min": 0.001293813574323635 + } + ] + }, + { + "a1": 0.07486023123142863, + "a2": -0.056984119959663924, + "fit_check_at_lw": { + "actual": 0.0037470717889539515, + "in_fit": false, + "pred": 0.0037467996364556446 + }, + "lamwin": 0.05211827956989248, + "n": 96, + "name": "theta*_window", + "points": [ + { + "agg": 0.0037470717889539515, + "lambda": 0.05211827956989248, + "max_marginal": 0.2686584707723108, + "per_i_min": 0.002400095876036172 + }, + { + "agg": 0.0019120965241105094, + "lambda": 0.02605913978494624, + "max_marginal": 0.26820586078534714, + "per_i_min": 0.0012435791433416873 + }, + { + "agg": 0.0009657224385259264, + "lambda": 0.01302956989247312, + "max_marginal": 0.2679812378601986, + "per_i_min": 0.0006327242244260241 + } + ] + }, + { + "a1": 0.06479531768625109, + "a2": -0.06006271172083667, + "fit_check_at_lw": { + "actual": 0.0019432202364606898, + "in_fit": false, + "pred": 0.0019431538076781596 + }, + "lamwin": 0.030872611464968155, + "n": 160, + "name": "theta*_window", + "points": [ + { + "agg": 0.0019432202364606898, + "lambda": 0.030872611464968155, + "max_marginal": 0.26802610453023695, + "per_i_min": 0.0014482677902202118 + }, + { + "agg": 0.0009858886188387423, + "lambda": 0.015436305732484078, + "max_marginal": 0.2677622454805284, + "per_i_min": 0.0007393955445394208 + }, + { + "agg": 0.0004965222381692867, + "lambda": 0.007718152866242039, + "max_marginal": 0.26763088780674327, + "per_i_min": 0.00037352370459895444 + } + ] + }, + { + "a1": 0.06026246786943206, + "a2": -0.06118181249487241, + "fit_check_at_lw": { + "actual": 0.0012922545161923378, + "in_fit": true, + "pred": 0.0012922545161923378 + }, + "lamwin": 0.02193212669683258, + "n": 224, + "name": "theta*_window", + "points": [ + { + "agg": 0.0012922545161923378, + "lambda": 0.02193212669683258, + "max_marginal": 0.2676130466340813, + "per_i_min": 0.0010305373992697993 + }, + { + "agg": 0.0006534846491421311, + "lambda": 0.01096606334841629, + "max_marginal": 0.26742793613919125, + "per_i_min": 0.0005229460542855949 + } + ] + }, + { + "a1": 0.056645694127628717, + "a2": -0.062006786146967745, + "fit_check_at_lw": { + "actual": 0.0008516285523159865, + "in_fit": true, + "pred": 0.0008516285523159865 + }, + "lamwin": 0.015290220820189276, + "n": 320, + "name": "theta*_window", + "points": [ + { + "agg": 0.0008516285523159865, + "lambda": 0.015290220820189276, + "max_marginal": 0.2671151765800672, + "per_i_min": 0.0007137365613028111 + }, + { + "agg": 0.00042943843101008215, + "lambda": 0.007645110410094638, + "max_marginal": 0.26698832373233494, + "per_i_min": 0.0003605764646636674 + } + ] + }, + { + "a1": 0.1751143939186219, + "a2": -0.02353120301995371, + "fit_check_at_lw": { + "actual": 0.01858716656824517, + "in_fit": false, + "pred": 0.018588764371728596 + }, + "lamwin": 0.10771111111111112, + "n": 48, + "name": "raw_window", + "points": [ + { + "agg": 0.01858716656824517, + "lambda": 0.10771111111111112, + "max_marginal": 0.2867542294354933, + "per_i_min": 0.0035375930597663363 + }, + { + "agg": 0.009362632578063041, + "lambda": 0.05385555555555556, + "max_marginal": 0.2859381905395171, + "per_i_min": 0.0018653986843678097 + }, + { + "agg": 0.004698378887081206, + "lambda": 0.02692777777777778, + "max_marginal": 0.28553616854862796, + "per_i_min": 0.0009571314374150867 + } + ] + }, + { + "a1": 0.11041190315309547, + "a2": -0.02750595583836855, + "fit_check_at_lw": { + "actual": 0.005679785621864555, + "in_fit": false, + "pred": 0.005679763594146985 + }, + "lamwin": 0.05211827956989248, + "n": 96, + "name": "raw_window", + "points": [ + { + "agg": 0.005679785621864555, + "lambda": 0.05211827956989248, + "max_marginal": 0.3552462326379544, + "per_i_min": 0.001176260369865241 + }, + { + "agg": 0.002858560507630977, + "lambda": 0.02605913978494624, + "max_marginal": 0.35445552628108656, + "per_i_min": 0.0006032660370033152 + }, + { + "agg": 0.0014339499314548598, + "lambda": 0.01302956989247312, + "max_marginal": 0.3540628724630177, + "per_i_min": 0.0003054452575243921 + } + ] + }, + { + "a1": 0.3938418220742915, + "a2": 0.026795335669205844, + "fit_check_at_lw": { + "actual": 0.012184379508460816, + "in_fit": false, + "pred": 0.012184464672012708 + }, + "lamwin": 0.030872611464968155, + "n": 160, + "name": "raw_window", + "points": [ + { + "agg": 0.012184379508460816, + "lambda": 0.030872611464968155, + "max_marginal": 0.29165610872224224, + "per_i_min": 0.00045782569525010947 + }, + { + "agg": 0.006085847555891856, + "lambda": 0.015436305732484078, + "max_marginal": 0.29157555892960924, + "per_i_min": 0.0002323064959658533 + }, + { + "agg": 0.0030413275829173038, + "lambda": 0.007718152866242039, + "max_marginal": 0.29153542968958573, + "per_i_min": 0.00011700636032015155 + } + ] + }, + { + "a1": 0.30447471660945086, + "a2": 0.004152275210295657, + "fit_check_at_lw": { + "actual": 0.00667977538053119, + "in_fit": true, + "pred": 0.0066797753805311905 + }, + "lamwin": 0.02193212669683258, + "n": 224, + "name": "raw_window", + "points": [ + { + "agg": 0.00667977538053119, + "lambda": 0.02193212669683258, + "max_marginal": 0.28390865965177403, + "per_i_min": 0.00023214999908315742 + }, + { + "agg": 0.0033393883602979655, + "lambda": 0.01096606334841629, + "max_marginal": 0.28385231038688497, + "per_i_min": 0.00011725454060178796 + } + ] + }, + { + "a1": 0.20757558590368316, + "a2": -0.012636676118997455, + "fit_check_at_lw": { + "actual": 0.0031709222060619493, + "in_fit": true, + "pred": 0.0031709222060619484 + }, + "lamwin": 0.015290220820189276, + "n": 320, + "name": "raw_window", + "points": [ + { + "agg": 0.0031709222060619493, + "lambda": 0.015290220820189276, + "max_marginal": 0.2730504932921306, + "per_i_min": 0.00010839278269737102 + }, + { + "agg": 0.0015861996878523583, + "lambda": 0.007645110410094638, + "max_marginal": 0.2730156226905197, + "per_i_min": 5.455704298863023e-05 + } + ] + }, + { + "agg": 0.005490483893613277, + "lambda": 0.10771111111111112, + "max_marginal": 0.37495690163435325, + "n": 48, + "stage": "window_opt", + "theta": { + "A1": 14.589171969742202, + "A2": 20.195701273393574, + "B0": 13.573329717299568, + "B1": 44.617694492699364, + "B2": 10.527521073256187, + "dil": 18.776813062766777, + "light": 2.7507571709706416e-06, + "resp": 2.3180607047139826e-07 + } + }, + { + "agg": 0.002163711119661682, + "from_n": 48, + "lambda": 0.05211827956989248, + "max_marginal": 0.29978940357156825, + "n": 96, + "stage": "window_transfer", + "theta": { + "A1": 14.589171969742202, + "A2": 20.195701273393574, + "B0": 13.573329717299568, + "B1": 44.617694492699364, + "B2": 10.527521073256187, + "dil": 31.622776601683793, + "light": 2.7507571709706416e-06, + "resp": 2.3180607047139826e-07 + } + }, + { + "agg": 0.0009001132611483325, + "from_n": 48, + "lambda": 0.030872611464968155, + "max_marginal": 0.2992454410424063, + "n": 160, + "stage": "window_transfer", + "theta": { + "A1": 14.589171969742202, + "A2": 20.195701273393574, + "B0": 13.573329717299568, + "B1": 44.617694492699364, + "B2": 10.527521073256187, + "dil": 31.622776601683793, + "light": 2.7507571709706416e-06, + "resp": 2.3180607047139826e-07 + } + }, + { + "agg": 0.0005210760592274509, + "from_n": 48, + "lambda": 0.02193212669683258, + "max_marginal": 0.29901781666789784, + "n": 224, + "stage": "window_transfer", + "theta": { + "A1": 14.589171969742202, + "A2": 20.195701273393574, + "B0": 13.573329717299568, + "B1": 44.617694492699364, + "B2": 10.527521073256187, + "dil": 31.622776601683793, + "light": 2.7507571709706416e-06, + "resp": 2.3180607047139826e-07 + } + }, + { + "agg": 0.0003007496980956152, + "from_n": 48, + "lambda": 0.015290220820189276, + "max_marginal": 0.29884909278108107, + "n": 320, + "stage": "window_transfer", + "theta": { + "A1": 14.589171969742202, + "A2": 20.195701273393574, + "B0": 13.573329717299568, + "B1": 44.617694492699364, + "B2": 10.527521073256187, + "dil": 31.622776601683793, + "light": 2.7507571709706416e-06, + "resp": 2.3180607047139826e-07 + } + }, + { + "agg": 0.0010204756992273473, + "lambda": 0.05211827956989248, + "max_marginal": 0.3737643611952598, + "n": 96, + "stage": "window_opt", + "theta": { + "A1": 16.27834869934754, + "A2": 33.79879931920271, + "B0": 6.235342596011143, + "B1": 33.99020922046475, + "B2": 13.772041802808104, + "dil": 10.480390210584487, + "light": 8.532373336370841e-07, + "resp": 1.2737714808601843e-05 + } + }, + { + "agg": 0.0007325755275602344, + "from_n": 96, + "lambda": 0.030872611464968155, + "max_marginal": 0.2530864766339607, + "n": 160, + "stage": "window_transfer", + "theta": { + "A1": 16.27834869934754, + "A2": 33.79879931920271, + "B0": 6.235342596011143, + "B1": 33.99020922046475, + "B2": 13.772041802808104, + "dil": 31.622776601683793, + "light": 8.532373336370841e-07, + "resp": 1.2737714808601843e-05 + } + }, + { + "agg": 0.000385074379901331, + "from_n": 96, + "lambda": 0.02193212669683258, + "max_marginal": 0.2529731047626092, + "n": 224, + "stage": "window_transfer", + "theta": { + "A1": 16.27834869934754, + "A2": 33.79879931920271, + "B0": 6.235342596011143, + "B1": 33.99020922046475, + "B2": 13.772041802808104, + "dil": 31.622776601683793, + "light": 8.532373336370841e-07, + "resp": 1.2737714808601843e-05 + } + }, + { + "agg": 0.00019708959706916848, + "from_n": 96, + "lambda": 0.015290220820189276, + "max_marginal": 0.252891248755056, + "n": 320, + "stage": "window_transfer", + "theta": { + "A1": 16.27834869934754, + "A2": 33.79879931920271, + "B0": 6.235342596011143, + "B1": 33.99020922046475, + "B2": 13.772041802808104, + "dil": 31.622776601683793, + "light": 8.532373336370841e-07, + "resp": 1.2737714808601843e-05 + } + }, + { + "agg": 1.6584238533584797e-05, + "lambda": 0.00081434811827957, + "lamwin": 0.05211827956989248, + "max_marginal": 0.3748178097789431, + "n": 96, + "stage": "free_window_climb", + "theta": { + "A1": 25.18131451744078, + "A2": 43.94973843469965, + "B0": 6.246506422490582, + "B1": 35.92476544634052, + "B2": 22.556998696301168, + "dil": 17.291425206253994, + "light": 0.008736245666839927, + "resp": 0.0005625081640755257 + } + } + ] +} diff --git a/problems/union-closed/data/gap1c_partE.json b/problems/union-closed/data/gap1c_partE.json new file mode 100644 index 0000000..d4f8935 --- /dev/null +++ b/problems/union-closed/data/gap1c_partE.json @@ -0,0 +1,40 @@ +{ + "rows": [ + { + "agg": 0.19016834584765038, + "dil": 80.0, + "max_marginal": 0.2605082762017913, + "n": 160 + }, + { + "agg": 0.15042029400883875, + "dil": 80.0, + "max_marginal": 0.2959808061161424, + "n": 192 + }, + { + "agg": 0.12317569634718278, + "dil": 80.0, + "max_marginal": 0.3300249700672521, + "n": 224 + }, + { + "agg": 0.10374478518633681, + "dil": 80.0, + "max_marginal": 0.3624750591427097, + "n": 256 + }, + { + "agg": 0.21364556553172936, + "dil": 160.0, + "max_marginal": 0.25854895138651934, + "n": 288 + }, + { + "agg": 0.18053712197718091, + "dil": 160.0, + "max_marginal": 0.25222736105650856, + "n": 320 + } + ] +} diff --git a/problems/union-closed/data/gap1c_partF.json b/problems/union-closed/data/gap1c_partF.json new file mode 100644 index 0000000..ac1e7ec --- /dev/null +++ b/problems/union-closed/data/gap1c_partF.json @@ -0,0 +1,80 @@ +{ + "rows": [ + { + "agg_hi_float": 1.8446690053408297, + "agg_lo_float": 1.8446690053408297, + "dich_ok": true, + "in_regime": true, + "kind": "mm", + "max_marg_float": 0.31777018538674157, + "mm_hi": "-0.022254658205655785176", + "mm_hi_float": -0.022254658205655784, + "mm_lo": "-0.022254658205655785177", + "mm_lo_float": -0.022254658205655784, + "n": 7, + "name": "mm_witness_t181/16", + "t": "181/16", + "verdict": "CERTIFIED NEGATIVE (KILL)" + }, + { + "agg_hi_float": -0.0007598651858441661, + "agg_lo_float": -0.0007598651858441661, + "dich_ok": true, + "in_regime": true, + "kind": "mm", + "max_marg_float": 0.30866648372421185, + "mm_hi": "-0.000381941148389669057", + "mm_hi_float": -0.00038194114838966905, + "mm_lo": "-0.000381941148389669058", + "mm_lo_float": -0.00038194114838966905, + "n": 96, + "name": "mm_theta*_n96_t4", + "t": "4", + "verdict": "CERTIFIED NEGATIVE (KILL)" + }, + { + "agg_hi_float": -0.02422590867196798, + "agg_lo_float": -0.02422590867196798, + "dich_ok": true, + "in_regime": true, + "kind": "mm", + "max_marg_float": 0.30718971680164464, + "mm_hi": "-0.000222999891309132913", + "mm_hi_float": -0.0002229998913091329, + "mm_lo": "-0.000222999891309132914", + "mm_lo_float": -0.0002229998913091329, + "n": 160, + "name": "mm_theta*_n160_t4", + "t": "4", + "verdict": "CERTIFIED NEGATIVE (KILL)" + }, + { + "agg_hi": "+0.002092188746927237207", + "agg_hi_float": 0.002092188746927237, + "agg_lo": "+0.002092188746927237206", + "agg_lo_float": 0.002092188746927237, + "dich_ok": true, + "in_regime": true, + "kind": "plain", + "max_marg_float": 0.2682492610858563, + "n": 96, + "name": "window_theta*_n96_t51/50", + "t": "51/50", + "verdict": "CERTIFIED POSITIVE" + }, + { + "agg_hi": "+0.001802171374037480778", + "agg_hi_float": 0.0018021713740374809, + "agg_lo": "+0.001802171374037480777", + "agg_lo_float": 0.0018021713740374809, + "dich_ok": true, + "in_regime": true, + "kind": "plain", + "max_marg_float": 0.2679866363283877, + "n": 160, + "name": "window_theta*_n160_t51/50", + "t": "51/50", + "verdict": "CERTIFIED POSITIVE" + } + ] +} diff --git a/problems/union-closed/data/gap1c_partR.json b/problems/union-closed/data/gap1c_partR.json new file mode 100644 index 0000000..fa3b496 --- /dev/null +++ b/problems/union-closed/data/gap1c_partR.json @@ -0,0 +1,289 @@ +{ + "friendly_genres": [ + { + "agg_plain": 0.3841499232776119, + "lambda": 1.0, + "max_marginal": 0.4066125449636509, + "mm_abs": 0.13623305601655214, + "mm_der": 0.004558279680878707, + "mm_sec": -0.0009687593235662646, + "n": 8, + "name": "d0_bern" + }, + { + "agg_plain": 0.09403766499305333, + "lambda": 2.0, + "max_marginal": 0.7280075832252616, + "mm_abs": 0.05646516984849629, + "mm_der": 0.046354162968939294, + "mm_sec": 0.00322078593354572, + "n": 8, + "name": "d0_bern" + }, + { + "agg_plain": 0.018238889177829426, + "lambda": 1.0, + "max_marginal": 0.3990511521968525, + "mm_abs": 0.016019970079520024, + "mm_der": 0.015842683227956973, + "mm_sec": 0.0003803768417029619, + "n": 8, + "name": "bern_mix" + }, + { + "agg_plain": 0.023169969403827245, + "lambda": 2.0, + "max_marginal": 0.5738712659102553, + "mm_abs": 0.021219827979754868, + "mm_der": 0.02121903883382601, + "mm_sec": 0.0011649231915104462, + "n": 8, + "name": "bern_mix" + }, + { + "agg_plain": 0.1861980959467007, + "lambda": 1.0, + "max_marginal": 0.37617173769538453, + "mm_abs": 0.3479127544947376, + "mm_der": 0.3211273851797505, + "mm_sec": 0.006191266224099342, + "n": 8, + "name": "smoothed_slice" + }, + { + "agg_plain": 0.4375105582210917, + "lambda": 2.0, + "max_marginal": 0.377230090648015, + "mm_abs": 0.7519336894423406, + "mm_der": 0.5901078677011317, + "mm_sec": 0.007020234759847104, + "n": 8, + "name": "smoothed_slice" + }, + { + "agg_plain": 0.009171443750129071, + "lambda": 1.0, + "max_marginal": 0.60824372853355, + "mm_abs": 0.009043613074406504, + "mm_der": 0.009043613074406504, + "mm_sec": 0.0006916955457095046, + "n": 6, + "name": "random_dense_0" + }, + { + "agg_plain": 0.026496914439104124, + "lambda": 2.0, + "max_marginal": 0.712177072528574, + "mm_abs": 0.026583516331252978, + "mm_der": 0.026583516331252978, + "mm_sec": 0.001964985769783403, + "n": 6, + "name": "random_dense_0" + }, + { + "agg_plain": 0.005482878415772799, + "lambda": 1.0, + "max_marginal": 0.6294251469148745, + "mm_abs": 0.00554216303649795, + "mm_der": 0.00554216303649795, + "mm_sec": 0.0004364488557651157, + "n": 6, + "name": "random_dense_1" + }, + { + "agg_plain": 0.01735495531496244, + "lambda": 2.0, + "max_marginal": 0.7387009566493377, + "mm_abs": 0.017485592896141378, + "mm_der": 0.017485592896141378, + "mm_sec": 0.0013495480269125623, + "n": 6, + "name": "random_dense_1" + }, + { + "agg_plain": 0.01009828855657687, + "lambda": 1.0, + "max_marginal": 0.6244447396415707, + "mm_abs": 0.009527478923182509, + "mm_der": 0.009527478923182509, + "mm_sec": 0.0006898364290005077, + "n": 6, + "name": "random_dense_2" + }, + { + "agg_plain": 0.032316355286211296, + "lambda": 2.0, + "max_marginal": 0.7604748130633303, + "mm_abs": 0.03206512291301736, + "mm_der": 0.03206512291301736, + "mm_sec": 0.0023416859114849162, + "n": 6, + "name": "random_dense_2" + } + ], + "perturbations": { + "n_fail": 0, + "rows": [ + { + "max_marginal": 0.31347657197132645, + "mm_sec": -0.02223561813112463 + }, + { + "max_marginal": 0.31714128846263123, + "mm_sec": -0.02245033290669669 + }, + { + "max_marginal": 0.3137766119275843, + "mm_sec": -0.02202490243382578 + }, + { + "max_marginal": 0.3129321936635126, + "mm_sec": -0.021827229145357726 + }, + { + "max_marginal": 0.3245865503565605, + "mm_sec": -0.02279142664511115 + }, + { + "max_marginal": 0.31786332648592625, + "mm_sec": -0.02182008743123427 + }, + { + "max_marginal": 0.30870484029871725, + "mm_sec": -0.022202790791563197 + }, + { + "max_marginal": 0.31682102518192046, + "mm_sec": -0.022608625975043595 + }, + { + "max_marginal": 0.31417213855253234, + "mm_sec": -0.022194990300345805 + }, + { + "max_marginal": 0.3162648002960624, + "mm_sec": -0.02263265038259147 + }, + { + "max_marginal": 0.3213213213328809, + "mm_sec": -0.02193247525011156 + }, + { + "max_marginal": 0.32341846997714513, + "mm_sec": -0.022208308393131364 + }, + { + "max_marginal": 0.3203660431832419, + "mm_sec": -0.022040557294252278 + }, + { + "max_marginal": 0.31260490191368495, + "mm_sec": -0.021956677965274476 + }, + { + "max_marginal": 0.3168396983801124, + "mm_sec": -0.021895060901413077 + }, + { + "max_marginal": 0.32196470870303995, + "mm_sec": -0.022157556510967312 + }, + { + "max_marginal": 0.3081230482296867, + "mm_sec": -0.022083668998029427 + }, + { + "max_marginal": 0.307702945749723, + "mm_sec": -0.022329658823909047 + }, + { + "max_marginal": 0.3205194895235831, + "mm_sec": -0.022200951704938767 + }, + { + "max_marginal": 0.30649991524444975, + "mm_sec": -0.022364681934444 + } + ] + }, + "product_anchors": [ + { + "lambda": 2.0, + "mm_sec": 8.321220340288123e-17, + "n": 6, + "p": 0.3 + }, + { + "lambda": 1.0, + "mm_sec": -2.249553961127159e-16, + "n": 8, + "p": 0.38 + } + ], + "tidy_2digit": { + "max_marginal": 0.3129819246529253, + "mm_sec": -0.022274080245055162 + }, + "witness_lambda_profile": [ + { + "lambda": 0.25, + "mm_abs": 0.14581221929937546, + "mm_der": 0.03337309478318317, + "mm_sec": 0.0005276399777154498 + }, + { + "lambda": 0.5, + "mm_abs": 0.2918830646001804, + "mm_der": 0.04619334857905371, + "mm_sec": 0.0006210320281791468 + }, + { + "lambda": 1.0, + "mm_abs": 0.5875241445896343, + "mm_der": 0.011515651932285194, + "mm_sec": -0.0007837257170032087 + }, + { + "lambda": 1.5, + "mm_abs": 0.8926651234051399, + "mm_der": -0.0942059906321367, + "mm_sec": -0.004412915219666583 + }, + { + "lambda": 2.0, + "mm_abs": 1.212369263355493, + "mm_der": -0.24943216995679324, + "mm_sec": -0.009678503756700377 + }, + { + "lambda": 2.5, + "mm_abs": 1.5507063011195057, + "mm_der": -0.43721495522390613, + "mm_sec": -0.015293204656759585 + }, + { + "lambda": 3.0, + "mm_abs": 1.9213171626616994, + "mm_der": -0.6495821563393758, + "mm_sec": -0.019811980562546908 + }, + { + "lambda": 3.5, + "mm_abs": 2.3543277749368867, + "mm_der": -0.8955419815884347, + "mm_sec": -0.022255048835182943 + }, + { + "lambda": 4.0, + "mm_abs": 2.909588328563616, + "mm_der": -1.2105993037252338, + "mm_sec": -0.02239453929848443 + }, + { + "lambda": 5.0, + "mm_abs": 3.5642131548635643, + "mm_der": -1.36986514988082, + "mm_sec": -0.017772380289434307 + } + ] +} diff --git a/problems/union-closed/data/gap1c_robust.log b/problems/union-closed/data/gap1c_robust.log new file mode 100644 index 0000000..dd27317 --- /dev/null +++ b/problems/union-closed/data/gap1c_robust.log @@ -0,0 +1,33 @@ +[ 0s] [A] witness plain aggregate (lam 3.5): +1.844754 (013 certified +1.844669) +[ 0s] [A] census_mm plain-agg cross-check: +1.844754 (diff 2.22e-16) +[ 3s] [A] theta* ladder n=96 lam=2 plain aggregate: -0.000759865 (016 certified -0.000759865) +[ 3s] [A] Plackett round-trip on 12 witness rows: max |z_rho(x,y,OR) - z~| = 1.55e-15 +[ 3s] [A] secant vs derivative (dl=1e-3): 0.000027704 vs 0.000027709 (rel diff 2.0e-04) +[ 3s] [A] MM padding invariance MU(20,1) vs witness: diff 0.00e+00 +[ 3s] [A] anchors ALL PASS +[ 3s] [R] product Bern(0.3)^6 lam=2.0: MM_sec +8.3e-17 +[ 3s] [R] product Bern(0.38)^8 lam=1.0: MM_sec -2.2e-16 +[ 3s] [R] d0_bern lam=1.0: MM_sec -0.00097 MM_der +0.00456 (mm 0.407) +[ 3s] [R] d0_bern lam=2.0: MM_sec +0.00322 MM_der +0.04635 (mm 0.728) +[ 4s] [R] bern_mix lam=1.0: MM_sec +0.00038 MM_der +0.01584 (mm 0.399) +[ 4s] [R] bern_mix lam=2.0: MM_sec +0.00116 MM_der +0.02122 (mm 0.574) +[ 4s] [R] smoothed_slice lam=1.0: MM_sec +0.00619 MM_der +0.32113 (mm 0.376) +[ 4s] [R] smoothed_slice lam=2.0: MM_sec +0.00702 MM_der +0.59011 (mm 0.377) +[ 4s] [R] random_dense_0 lam=1.0: MM_sec +0.00069 MM_der +0.00904 (mm 0.608) +[ 4s] [R] random_dense_0 lam=2.0: MM_sec +0.00196 MM_der +0.02658 (mm 0.712) +[ 4s] [R] random_dense_1 lam=1.0: MM_sec +0.00044 MM_der +0.00554 (mm 0.629) +[ 4s] [R] random_dense_1 lam=2.0: MM_sec +0.00135 MM_der +0.01749 (mm 0.739) +[ 4s] [R] random_dense_2 lam=1.0: MM_sec +0.00069 MM_der +0.00953 (mm 0.624) +[ 4s] [R] random_dense_2 lam=2.0: MM_sec +0.00234 MM_der +0.03207 (mm 0.760) +[ 4s] [R] witness 20 x 3% perturbations: 0 failures; MM_sec in [-0.02279, -0.02182] +[ 4s] [R] 2-digit tidy witness: MM_sec -0.02227 (mm 0.3130) +[ 4s] [R] witness lam=0.25: MM_sec +0.000528 MM_der +0.033373 MM_abs +0.145812 +[ 4s] [R] witness lam=0.50: MM_sec +0.000621 MM_der +0.046193 MM_abs +0.291883 +[ 4s] [R] witness lam=1.00: MM_sec -0.000784 MM_der +0.011516 MM_abs +0.587524 +[ 4s] [R] witness lam=1.50: MM_sec -0.004413 MM_der -0.094206 MM_abs +0.892665 +[ 4s] [R] witness lam=2.00: MM_sec -0.009679 MM_der -0.249432 MM_abs +1.212369 +[ 4s] [R] witness lam=2.50: MM_sec -0.015293 MM_der -0.437215 MM_abs +1.550706 +[ 4s] [R] witness lam=3.00: MM_sec -0.019812 MM_der -0.649582 MM_abs +1.921317 +[ 4s] [R] witness lam=3.50: MM_sec -0.022255 MM_der -0.895542 MM_abs +2.354328 +[ 4s] [R] witness lam=4.00: MM_sec -0.022395 MM_der -1.210599 MM_abs +2.909588 +[ 4s] [R] witness lam=5.00: MM_sec -0.017772 MM_der -1.369865 MM_abs +3.564213 diff --git a/problems/union-closed/data/gap1c_run.log b/problems/union-closed/data/gap1c_run.log new file mode 100644 index 0000000..83d57ff --- /dev/null +++ b/problems/union-closed/data/gap1c_run.log @@ -0,0 +1,68 @@ +[ 0s] [A] witness plain aggregate (lam 3.5): +1.844754 (013 certified +1.844669) +[ 0s] [A] census_mm plain-agg cross-check: +1.844754 (diff 2.22e-16) +[ 2s] [A] theta* ladder n=96 lam=2 plain aggregate: -0.000759865 (016 certified -0.000759865) +[ 2s] [A] Plackett round-trip on 12 witness rows: max |z_rho(x,y,OR) - z~| = 1.55e-15 +[ 2s] [A] secant vs derivative (dl=1e-3): 0.000027704 vs 0.000027709 (rel diff 2.0e-04) +[ 2s] [A] MM padding invariance MU(20,1) vs witness: diff 0.00e+00 +[ 2s] [A] anchors ALL PASS +[ 2s] [B] witness (n=7, lam=3.5): plain +1.8448 MM_sec -0.022255 MM_der -0.895542 MM_abs +2.354328 +[ 3s] [B] raw ladder n= 16 lam=2: plain +0.633033 MM_sec -5.702361e-03 MM_der -2.390113e-01 MM_abs +1.239965e+00 (mm 0.242) +[ 3s] [B] raw ladder n= 32 lam=2: plain +0.353040 MM_sec -4.382961e-03 MM_der -3.915036e-01 MM_abs +1.156959e+00 (mm 0.283) +[ 3s] [B] raw ladder n= 48 lam=2: plain +0.239794 MM_sec -3.782969e-03 MM_der -5.373214e-01 MM_abs +1.085607e+00 (mm 0.322) +[ 3s] [B] raw ladder n= 64 lam=2: plain +0.180142 MM_sec -3.400008e-03 MM_der -6.193039e-01 MM_abs +1.071119e+00 (mm 0.359) +[ 6s] [B] raw ladder n= 96 lam=2: plain +1.163014 MM_sec +6.719528e-03 MM_der +9.856109e-01 MM_abs +1.514688e+00 (mm 0.311) +[ 12s] [B] raw ladder n=128 lam=2: plain +0.986210 MM_sec +6.054830e-03 MM_der +9.701218e-01 MM_abs +1.503923e+00 (mm 0.307) +[ 25s] [B] raw ladder n=160 lam=2: plain +0.828164 MM_sec +5.245485e-03 MM_der +9.523977e-01 MM_abs +1.495748e+00 (mm 0.304) +[ 25s] [B] theta* ladder n= 16 lam=2: plain +0.296990 MM_sec -2.415558e-03 MM_der -1.766320e-01 MM_abs +9.641139e-01 (mm 0.311) +[ 25s] [B] theta* ladder n= 32 lam=2: plain +0.116826 MM_sec -1.184227e-03 MM_der -1.755895e-01 MM_abs +9.650983e-01 (mm 0.310) +[ 25s] [B] theta* ladder n= 48 lam=2: plain +0.057832 MM_sec -7.813112e-04 MM_der -1.745147e-01 MM_abs +9.660837e-01 (mm 0.310) +[ 26s] [B] theta* ladder n= 64 lam=2: plain +0.028503 MM_sec -5.812057e-04 MM_der -1.734073e-01 MM_abs +9.670700e-01 (mm 0.309) +[ 28s] [B] theta* ladder n= 96 lam=2: plain -0.000760 MM_sec -3.819411e-04 MM_der -1.710923e-01 MM_abs +9.690449e-01 (mm 0.309) +[ 34s] [B] theta* ladder n=128 lam=2: plain -0.015406 MM_sec -2.825735e-04 MM_der -1.686405e-01 MM_abs +9.710224e-01 (mm 0.308) +[ 51s] [B] theta* ladder n=160 lam=2: plain -0.024226 MM_sec -2.229999e-04 MM_der -1.660474e-01 MM_abs +9.730018e-01 (mm 0.307) +[ 54s] [B] theta* ladder n=96 lam=0.5: plain +0.023912 MM_sec -3.660909e-05 MM_abs +1.987985e-01 +[ 57s] [B] theta* ladder n=96 lam=1.0: plain +0.025207 MM_sec -1.443052e-04 MM_abs +4.189173e-01 +[ 60s] [B] theta* ladder n=96 lam=1.5: plain +0.013163 MM_sec -2.798722e-04 MM_abs +6.606552e-01 +[ 63s] [B] theta* ladder n=96 lam=2.0: plain -0.000760 MM_sec -3.819411e-04 MM_abs +9.690449e-01 +[ 66s] [B] theta* ladder n=96 lam=2.5: plain -0.005632 MM_sec -4.120991e-04 MM_abs +1.381158e+00 +[ 69s] [B] theta* ladder n=96 lam=3.0: plain +0.953119 MM_sec -5.992615e-03 MM_abs +2.362848e+00 +[ 72s] [B] theta* ladder n=96 lam=3.5: plain +1.016433 MM_sec -8.051331e-03 MM_abs +2.915241e+00 +[ 148s] [C] MM theta-opt n=48 lam=2.0: MM_abs +2.462548e-01 MM_sec -5.344705e-04 plain +0.4691 (mm 0.363) +[ 149s] [C] MM theta(2.0) at n=64: MM_abs +2.387101e-01 MM_sec -4.267747e-04 plain +0.4487 (mm 0.364) +[ 152s] [C] MM theta(2.0) at n=96: MM_abs +1.003748e+00 MM_sec -5.378083e-03 plain +0.5818 (mm 0.247) +[ 159s] [C] MM theta(2.0) at n=128: MM_abs +1.055445e+00 MM_sec -4.350515e-03 plain +0.5057 (mm 0.242) +[ 177s] [C] MM theta(2.0) at n=160: MM_abs +1.094052e+00 MM_sec -3.574522e-03 plain +0.4455 (mm 0.237) +[ 255s] [C] MM theta-opt n=48 lam=3.5: MM_abs +6.393336e-02 MM_sec -1.513089e-04 plain +2.4560 (mm 0.041) +[ 256s] [C] MM theta(3.5) at n=64: MM_abs +1.410008e-01 MM_sec -8.540340e-05 plain +2.5294 (mm 0.043) +[ 259s] [C] MM theta(3.5) at n=96: MM_abs +1.464061e-01 MM_sec -5.831230e-05 plain +2.5775 (mm 0.044) +[ 267s] [C] MM theta(3.5) at n=128: MM_abs +1.524140e-01 MM_sec -4.511369e-05 plain +2.5953 (mm 0.046) +[ 283s] [C] MM theta(3.5) at n=160: MM_abs +1.590280e-01 MM_sec -3.741645e-05 plain +2.6011 (mm 0.048) +[ 284s] [C] free climb n=8 lam=1.0: endpoint MM_abs +1.615935e-01 MM_sec -2.839242e-03 (mm 0.342) +[ 284s] [C] free climb n=8 lam=2.0: endpoint MM_abs +1.007614e-02 MM_sec -8.327292e-05 (mm 0.153) +[ 285s] [C] free climb n=8 lam=3.5: endpoint MM_abs +4.053895e-01 MM_sec -8.747597e-03 (mm 0.164) +[ 285s] [C] free climb n=8 lam=2.0: endpoint MM_abs +3.194872e+00 MM_sec +2.347615e-03 (mm 0.471) +[ 286s] [C] free climb n=8 lam=3.5: endpoint MM_abs +5.921476e+00 MM_sec +8.776165e-02 (mm 0.663) +[ 287s] [C] free climb n=8 lam=1.0: endpoint MM_abs +1.180410e+00 MM_sec +7.575692e-03 (mm 0.384) +[ 287s] [C] free climb n=8 lam=2.0: endpoint MM_abs +2.889770e+00 MM_sec +4.204783e-02 (mm 0.495) +[ 288s] [C] free climb n=8 lam=3.5: endpoint MM_abs +5.716586e+00 MM_sec +2.406190e-02 (mm 0.643) +[ 289s] [D] theta* n= 48 window lam=0.1077: A(lw) +1.005266e-02 A(lw/2) +5.167597e-03 a1 +9.85895e-02 a2 -4.89563e-02 a1*n +4.732 +[ 297s] [D] theta* n= 96 window lam=0.0521: A(lw) +3.747072e-03 A(lw/2) +1.912097e-03 a1 +7.48602e-02 a2 -5.69841e-02 a1*n +7.187 +[ 353s] [D] theta* n=160 window lam=0.0309: A(lw) +1.943220e-03 A(lw/2) +9.858886e-04 a1 +6.47953e-02 a2 -6.00627e-02 a1*n +10.367 +[ 466s] [D] theta* n=224 window lam=0.0219: A(lw) +1.292255e-03 A(lw/2) +6.534846e-04 a1 +6.02625e-02 a2 -6.11818e-02 a1*n +13.499 +[ 859s] [D] theta* n=320 window lam=0.0153: A(lw) +8.516286e-04 A(lw/2) +4.294384e-04 a1 +5.66457e-02 a2 -6.20068e-02 a1*n +18.127 +[ 860s] [D] raw n= 48 window lam=0.1077: A(lw) +1.858717e-02 A(lw/2) +9.362633e-03 a1 +1.75114e-01 a2 -2.35312e-02 a1*n +8.405 +[ 867s] [D] raw n= 96 window lam=0.0521: A(lw) +5.679786e-03 A(lw/2) +2.858561e-03 a1 +1.10412e-01 a2 -2.75060e-02 a1*n +10.600 +[ 909s] [D] raw n=160 window lam=0.0309: A(lw) +1.218438e-02 A(lw/2) +6.085848e-03 a1 +3.93842e-01 a2 +2.67953e-02 a1*n +63.015 +[ 1004s] [D] raw n=224 window lam=0.0219: A(lw) +6.679775e-03 A(lw/2) +3.339388e-03 a1 +3.04475e-01 a2 +4.15228e-03 a1*n +68.202 +[ 1460s] [D] raw n=320 window lam=0.0153: A(lw) +3.170922e-03 A(lw/2) +1.586200e-03 a1 +2.07576e-01 a2 -1.26367e-02 a1*n +66.424 +[ 1559s] [D] window theta-opt n=48 lam=0.1077: agg +5.490484e-03 (mm 0.375) +[ 1562s] [D] window theta(n=48) at n=96 lam=0.0521: agg +2.163711e-03 (mm 0.300) +[ 1579s] [D] window theta(n=48) at n=160 lam=0.0309: agg +9.001133e-04 (mm 0.299) +[ 1642s] [D] window theta(n=48) at n=224 lam=0.0219: agg +5.210761e-04 (mm 0.299) +[ 1858s] [D] window theta(n=48) at n=320 lam=0.0153: agg +3.007497e-04 (mm 0.299) +[ 2673s] [D] window theta-opt n=96 lam=0.0521: agg +1.020476e-03 (mm 0.374) +[ 2687s] [D] window theta(n=96) at n=160 lam=0.0309: agg +7.325755e-04 (mm 0.253) +[ 2757s] [D] window theta(n=96) at n=224 lam=0.0219: agg +3.850744e-04 (mm 0.253) +[ 2969s] [D] window theta(n=96) at n=320 lam=0.0153: agg +1.970896e-04 (mm 0.253) +[ 3496s] [D] free (theta, lam<=window) climb n=96: endpoint agg +1.658424e-05 at lam=0.0008 (window 0.0521) +[ 3496s] done (parts A-D) diff --git a/problems/union-closed/data/gap1csk_run.log b/problems/union-closed/data/gap1csk_run.log new file mode 100644 index 0000000..b7eefb7 --- /dev/null +++ b/problems/union-closed/data/gap1csk_run.log @@ -0,0 +1,40 @@ +[ 0s] [S0] log2 enclosure: 67 cases + additivity, max width 9.1e-30, ok=True +[ 4s] [S0] Plackett Newton root: 60 random (x,y,t), max width 6.6e-49, round-trip OR/t = 1 to 1e-9, ok=True +[ 4s] [S0] h2 enclosure: 41 cases ok=True +[ 4s] [S0] MU(7,1) from spec == 007 witness atom-for-atom: True +[ 4s] [S1] witness t=181/16 MM_sec in [-2.225465820565578e-02, -2.225465820565578e-02] width 5.9e-30 CERTIFIED NEGATIVE +[ 4s] [S1] plain aggregate in [+1.844669005341, +1.844669005341] (013/018: +1.844669005) +[ 4s] [S1] max marginal 0.3177701854 (exact < 38271/100000: True); dichotomy exact True; degenerate rows contribute 0 exactly True; rows 12 (12 distinct margins) +[ 4s] [S1] overlap with 018's enclosure [-0.022254658205655785177, ...176]: True +[ 5s] [S1] tidy 2-digit witness t=181/16: MM_sec in [-2.227367482e-02, -2.227367482e-02] CERTIFIED NEGATIVE (max marg 0.3130, in-regime True) +[ 5s] [S1] product anchor (w=3/7, n=5, t=4): MM_sec enclosure contains 0: True (width 9.3e-30); plain aggregate exactly 0: True +[ 5s] [S2] witness (n=7, lam=3.5) variant zoo: +[ 5s] [S2] plain +1.844754 +[ 5s] [S2] mm_sec -0.022255 +[ 5s] [S2] mm_der -0.895542 +[ 5s] [S2] mm_abs +2.354328 +[ 5s] [S2] mm_der_rlz -1.458123 +[ 5s] [S2] mm_abs_rlz +2.132730 +[ 5s] [S2] mm_absrho_rlz +1.808150 +[ 8s] [S2] theta* ladder n=96 lam=2 variant zoo: +[ 8s] [S2] plain -7.598652e-04 +[ 8s] [S2] mm_sec -3.819411e-04 +[ 8s] [S2] mm_der -1.710923e-01 +[ 8s] [S2] mm_abs +9.690449e-01 +[ 8s] [S2] mm_der_rlz -4.351719e-01 +[ 8s] [S2] mm_abs_rlz +9.339568e-01 +[ 8s] [S2] mm_absrho_rlz +8.801585e-01 +[ 8s] [S2] witness MM_sec near the crossing: 0.60:+5.18e-04 0.70:+3.29e-04 0.75:+2.00e-04 0.80:+4.95e-05 0.90:-3.21e-04 +[ 8s] [S3] witness floats: MM_abs +2.354328 (018 +2.354328 True), MM_der -0.895542 (018 -0.895542 True), MM_sec -0.022255 (018 -0.022255 True) +[ 10s] [S3] theta* ladder n=96 lam=2 (own tune_dil, dil=31.622777): plain -0.000760 (018 -0.000760), MM_sec -3.819411e-04 (018 -3.819e-4), MM_abs +0.969045 (018 +0.969), mm 0.309, ok=True +[ 12s] [S3] raw ladder n=96 lam=2 (own tune_dil, dil=251.4867): plain +1.163014, MM_sec +6.719528e-03, MM_der +9.856109e-01 <-- MM_sec/MM_der POSITIVE at n=96 (018 checkpoint agrees; 018 PROSE says negative at every n) +[ 12s] [S3] raw ladder n=64 lam=2: MM_sec -3.400008e-03 (negative branch, 018 -3.400e-3) +[ 14s] [S3] theta* window n=96 lam=0.052118: A +3.747072e-03 (018 +3.747e-3, ok=True), mm 0.269 +[ 14s] [S3] a1/a2 fit re-solve (theta*): n=48:a1*n=+4.732 n=96:a1*n=+7.187 n=160:a1*n=+10.367 n=224:a1*n=+13.499 n=320:a1*n=+18.127 +[ 14s] [S3] all 10 stored (a1, a2) rows re-derive: True; a1*n growth on theta* rows: [4.73, 7.19, 10.37, 13.5, 18.13] +[ 14s] [S3] perturbations seed=7: 0/20 failures, MM_sec in [-0.02295, -0.02164] +[ 14s] [S3] perturbations seed=424242: 0/20 failures, MM_sec in [-0.02258, -0.02165] +[ 14s] [S4] best_free endpoint (n=8, lam=2.0): my MM_abs +1.007614e-02 (018 +1.007614e-02), MM_sec -8.327292e-05, max marg 0.153275 (in-regime True), dich True, H 1.019 bits, match=True +[ 14s] [S4] golden parenthetical: at marginal 0.381966, x = 0.618034, x^2 = 0.381966 = the marginal itself (phi identity), NOT 1/2; z_1(x,x) = 1/2 at marginal 0.292893 +[ 14s] [S4] z_t(x,x) = 1/2 at lam=3.5 happens at element marginal 0.389113 (lambda-dependent; equals 0.38197 for no stated reason) +[ 14s] done diff --git a/problems/union-closed/data/gap1csk_s0.json b/problems/union-closed/data/gap1csk_s0.json new file mode 100644 index 0000000..2c22551 --- /dev/null +++ b/problems/union-closed/data/gap1csk_s0.json @@ -0,0 +1,8 @@ +{ + "h2_ok": true, + "ladder_matches_witness": true, + "log2_max_width": 9.099415014103269e-30, + "ok": true, + "root_max_width": 6.6038125725103085e-49, + "root_ok": true +} diff --git a/problems/union-closed/data/gap1csk_s1.json b/problems/union-closed/data/gap1csk_s1.json new file mode 100644 index 0000000..a4b2dc6 --- /dev/null +++ b/problems/union-closed/data/gap1csk_s1.json @@ -0,0 +1,31 @@ +{ + "product_anchor": { + "plain_exact_zero": true, + "width": 9.339369011143012e-30, + "zero_in": true + }, + "tidy": { + "deg_zero_ok": true, + "dich_ok": true, + "in_regime": true, + "max_marg": 0.3129834963071075, + "mm_hi": -0.022273674819994822, + "mm_lo": -0.022273674819994822, + "verdict": "CERTIFIED NEGATIVE" + }, + "witness": { + "agg_hi": 1.8446690053408297, + "agg_lo": 1.8446690053408297, + "deg_zero_ok": true, + "dich_ok": true, + "in_regime": true, + "max_marg": 0.31777018538674157, + "mm_hi": "-2.225465820565578415e-02", + "mm_lo": "-2.225465820565578415e-02", + "mm_lo_str": "-10202624923862705339657959575/458448960643656010139491366619", + "n_rows": 12, + "overlaps_018_cert": true, + "verdict": "CERTIFIED NEGATIVE", + "width": 5.885365816786243e-30 + } +} diff --git a/problems/union-closed/data/gap1csk_s2.json b/problems/union-closed/data/gap1csk_s2.json new file mode 100644 index 0000000..e52a95c --- /dev/null +++ b/problems/union-closed/data/gap1csk_s2.json @@ -0,0 +1,44 @@ +{ + "crossing_profile": [ + { + "lambda": 0.6, + "mm_sec": 0.0005180399159924245 + }, + { + "lambda": 0.7, + "mm_sec": 0.00032852524915859797 + }, + { + "lambda": 0.75, + "mm_sec": 0.00020034433796453277 + }, + { + "lambda": 0.8, + "mm_sec": 4.948754676230562e-05 + }, + { + "lambda": 0.9, + "mm_sec": -0.0003210116383112507 + } + ], + "ladder96": { + "max_marginal": 0.308666483724311, + "mm_abs": 0.9690449182509435, + "mm_abs_rlz": 0.9339568166659286, + "mm_absrho_rlz": 0.8801584832291622, + "mm_der": -0.17109234495587256, + "mm_der_rlz": -0.43517192441768526, + "mm_sec": -0.0003819411483890162, + "plain": -0.0007598651859861864 + }, + "witness": { + "max_marginal": 0.3177685743531096, + "mm_abs": 2.354327774936892, + "mm_abs_rlz": 2.132730000950444, + "mm_absrho_rlz": 1.8081502902080455, + "mm_der": -0.8955419815884379, + "mm_der_rlz": -1.4581226521538753, + "mm_sec": -0.0222550488351827, + "plain": 1.844754315091964 + } +} diff --git a/problems/union-closed/data/gap1csk_s3.json b/problems/union-closed/data/gap1csk_s3.json new file mode 100644 index 0000000..03b052c --- /dev/null +++ b/problems/union-closed/data/gap1csk_s3.json @@ -0,0 +1,128 @@ +{ + "fit_all_match": true, + "fit_rows": [ + { + "a1": 0.09858947695317884, + "a1n": 4.732294893752584, + "a2": -0.048956262221548634, + "match": true, + "n": 48, + "name": "theta*_window" + }, + { + "a1": 0.07486023123142863, + "a1n": 7.186582198217148, + "a2": -0.056984119959663924, + "match": true, + "n": 96, + "name": "theta*_window" + }, + { + "a1": 0.06479531768625109, + "a1n": 10.367250829800174, + "a2": -0.06006271172083667, + "match": true, + "n": 160, + "name": "theta*_window" + }, + { + "a1": 0.06026246786943206, + "a1n": 13.498792802752781, + "a2": -0.06118181249487241, + "match": true, + "n": 224, + "name": "theta*_window" + }, + { + "a1": 0.056645694127628717, + "a1n": 18.12662212084119, + "a2": -0.062006786146967745, + "match": true, + "n": 320, + "name": "theta*_window" + }, + { + "a1": 0.1751143939186219, + "a1n": 8.405490908093851, + "a2": -0.02353120301995371, + "match": true, + "n": 48, + "name": "raw_window" + }, + { + "a1": 0.11041190315309547, + "a1n": 10.599542702697166, + "a2": -0.02750595583836855, + "match": true, + "n": 96, + "name": "raw_window" + }, + { + "a1": 0.3938418220742915, + "a1n": 63.01469153188664, + "a2": 0.026795335669205844, + "match": true, + "n": 160, + "name": "raw_window" + }, + { + "a1": 0.30447471660945086, + "a1n": 68.202336520517, + "a2": 0.004152275210295657, + "match": true, + "n": 224, + "name": "raw_window" + }, + { + "a1": 0.20757558590368316, + "a1n": 66.4241874891786, + "a2": -0.012636676118997455, + "match": true, + "n": 320, + "name": "raw_window" + } + ], + "perturbations": { + "7": { + "max": -0.021637631382637666, + "min": -0.02294828937536602, + "n_fail": 0 + }, + "424242": { + "max": -0.021647443457434683, + "min": -0.02258071839030879, + "n_fail": 0 + } + }, + "raw64": { + "mm_sec": -0.0034000080267894807, + "plain": 0.180142390793786 + }, + "raw96": { + "dil": 251.48668593658707, + "max_marginal": 0.3110651388915359, + "mm_der": 0.9856109084724816, + "mm_sec": 0.006719527843867126, + "plain": 1.1630136355382952 + }, + "theta96": { + "dil": 31.622776601683793, + "max_marginal": 0.308666483724311, + "mm_abs": 0.9690449182509435, + "mm_sec": -0.0003819411483890162, + "ok": true, + "plain": -0.0007598651859861864 + }, + "window96": { + "agg": 0.0037470717889539562, + "lamwin": 0.05211827956989248, + "max_marginal": 0.2686584707723108, + "ok": true + }, + "witness": { + "mm_abs": 2.354327774936892, + "mm_der": -0.8955419815884379, + "mm_sec": -0.0222550488351827, + "ok": true + } +} diff --git a/problems/union-closed/data/gap1csk_s4.json b/problems/union-closed/data/gap1csk_s4.json new file mode 100644 index 0000000..d676228 --- /dev/null +++ b/problems/union-closed/data/gap1csk_s4.json @@ -0,0 +1,20 @@ +{ + "best_free": { + "H": 1.0189536306703266, + "dich_ok": true, + "in_regime": true, + "match": true, + "max_marginal": 0.15327507789699268, + "mm_abs": 0.010076135813205649, + "mm_sec": -8.327292197827584e-05 + }, + "golden": { + "barrier": 0.3819660112501051, + "claim_x2_equals_half": false, + "crossing_marginal_at_lam3.5": 0.38911286640098963, + "true_crossing_marginal_rho1": 0.2928932188134524, + "x2_at_barrier": 0.3819660112501052, + "x2_equals_marginal": true, + "x_at_barrier": 0.6180339887498949 + } +} diff --git a/problems/union-closed/explore/uc_gap1_candidates.py b/problems/union-closed/explore/uc_gap1_candidates.py new file mode 100644 index 0000000..3bd3f65 --- /dev/null +++ b/problems/union-closed/explore/uc_gap1_candidates.py @@ -0,0 +1,933 @@ +#!/usr/bin/env python3 +"""Attempt 018: run the MU(n, r) ladder adversary against BOTH surviving +Gap-1 candidates (016 leads 1 and 4; STATUS queue items 1 and 20). + +After 016/017 killed the plain i-aggregated odds-ratio control at large n, +two restatements survived. Each gets a falsifiable probe here, against the +one adversary genre known to kill at scale (unit replication with +adversarial shared weights), with the adversary allowed to re-optimize its +weights against the NEW statement (a positive without that step would be +worthless). + +Candidate I -- margin-modulated control (007 lead 2). The chain-rule cost +of an OR-deviation at a history (a, b) is the change it makes to the +history's Plackett gain h(z_rho(x, y)), where x, y are the conditional +zero-margins of the response bit at that history. Primary form tested +(assembly-exact, secant form): + + MM(mu, lam) = E_hist[ h2(z~) - h2(z_{2^lam}(x, y)) ] >=? 0 + +where z~ = realized conditional both-zero probability (so h2(z~) = +h2(z_OR(x, y)) exactly), and the expectation is mass-weighted over +nondegenerate histories of all i < n. By the mean-value theorem this IS +E[sigma * (log2 OR - lam)] with sigma the Plackett sensitivity of 007 +lead 2, evaluated on the secant; two pointwise-weighted variants +(signed derivative at the target, |derivative| at the target) are computed +alongside. Degenerate-margin histories are excluded by the same N_i +bookkeeping as the plain control (their sensitivity weight -> 0 anyway -- +that is the entire point of the candidate). + +Candidate II -- lambda-restricted control (017 C2; 016 lead 4). The plain +aggregate A(mu, lam) >= 0 restricted to the workable window +lam <= LAMWIN(n) = 4.847/(n-3) (the 009/011 lambda-window law). The kill +in 016 lives at lam in [2, 2.5], ~40x above the window at n = 96; this +probe measures the ladder INSIDE the window, with theta re-optimized at +window lambda, and fits the small-lambda expansion +A ~ a1(n)*lam + a2(n)*lam^2 to see whether first order (Theorem C's +perfect square, a1 >= 0) protects the window asymptotically. + +Parts: + A anchors: 007 witness aggregate; 016's theta* kill at n = 96 reproduced; + Plackett round-trip identities on real census rows; padding invariance. + B MM on the witness + per-history table; MM along the raw-weight and + theta*(lam=2) ladders, n up to 160; lambda profiles at n = 96. + C adversarial theta re-optimization AGAINST MM at n = 48 (lam 2 and 3.5), + transfer to n = 64..160; free-support spot climbs at n = 8 against MM. + D window probe: per-n ladder at lam = LAMWIN(n), LAMWIN/2, LAMWIN/4 with + theta re-optimized at window lambda (n = 48 and 96 opts, transfers to + 320); small-lambda coefficient fit a1(n), a2(n); free (theta, lam) + climb with lam <= LAMWIN as a constraint. + E (--ext) raw-witness-weight ladder extension past n = 128 (016 lead 3), + honest dilution-swept minimum, lam = 2. + F (--cert) exact rational certification at rational tilt t of the + decisive points found by B-D: plain aggregate via 016's exact path + (013's engine), MM via a new certified interval path (dyadic bisection + for z_t, directed log2 enclosures for h2), with (x, y)-level caching. + +Standard library only. Deterministic (fixed seeds, fixed step counts). +Default run (A-D) ~20-40 min; --ext and --cert are separate longer passes. + +Usage: + python problems/union-closed/explore/uc_gap1_candidates.py [--fast] + python problems/union-closed/explore/uc_gap1_candidates.py --ext + python problems/union-closed/explore/uc_gap1_candidates.py --cert +Checkpoints: problems/union-closed/data/gap1c_part[A-F]*.json +""" + +from __future__ import annotations + +import json +import math +import random +import sys +import time +from fractions import Fraction +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +DATA = ROOT / "data" +sys.path.insert(0, str(HERE)) + +import uc_or_avg as ORA # 007 census (float) +import uc_or_avg_skeptic as ORS # 013 exact machinery +from uc_agg_ctrl_probe import (agg_eval, mu_ladder, exact_aggregate, + dec_bound, MARG_TARGET) +from uc_agg_ctrl_probe2 import (mu_ladder_theta, tune_dil, THETA_WITNESS, + marginals_of_u) + +FAST = "--fast" in sys.argv +T0 = time.monotonic() + +LAMWIN_C = 4.847 # 009/011 lambda-window law: lam_max ~ LAMWIN_C/(n-3) + + +def lamwin(n: int) -> float: + return LAMWIN_C / (n - 3) + + +# theta* from 016 P2 (data/aggprobe2_partP2.json), the weights that force +# the certified kill at n >= 96, lam = 2 +THETA_STAR_L2 = { + "A1": 14.026917090492047, "A2": 22.622718089566355, + "B0": 13.573329717299568, "B1": 43.00957928858568, + "B2": 24.62436309125563, "dil": 21.044172077650057, + "light": 0.007943082225456255, "resp": 0.0017259962695652103, +} + + +def save(name: str, obj) -> None: + DATA.mkdir(exist_ok=True) + (DATA / name).write_text(json.dumps(obj, indent=1, sort_keys=True, + default=str) + "\n", encoding="utf-8") + + +def log(msg: str) -> None: + print(f"[{time.monotonic() - T0:7.0f}s] {msg}", flush=True) + + +# ====================================================================== +# Plackett helpers (zero-margin convention as in uc_pert/uc_couplings: +# x = P(A_i = 0 | hist), y = P(B_i = 0 | hist), z = P(both 0 | hist), +# rho = z(1-x-y+z) / ((x-z)(y-z))) +# ====================================================================== + +def h2(z: float) -> float: + """binary entropy in bits; 0 at the endpoints""" + if z <= 0.0 or z >= 1.0: + return 0.0 + return -z * math.log2(z) - (1.0 - z) * math.log2(1.0 - z) + + +def z_plackett(x: float, y: float, rho: float) -> float: + """Both-zero probability of the Plackett coupling with zero-margins + x, y and odds ratio rho (stable root of a z^2 - S z + rho x y = 0).""" + if rho == 1.0: + return x * y + a = rho - 1.0 + S = 1.0 + a * (x + y) + disc = S * S - 4.0 * a * rho * x * y + return 2.0 * rho * x * y / (S + math.sqrt(disc)) + + +def dh_dlam(x: float, y: float, rho: float) -> float: + """d/d lam of h2(z_{2^lam}(x, y)) at 2^lam = rho (signed). Vanishes as + a margin degenerates: dz/drho carries the factor (x-z)(y-z).""" + z = z_plackett(x, y, rho) + if z <= 0.0 or z >= 1.0 or z >= min(x, y) or z <= max(0.0, x + y - 1.0): + return 0.0 + denom = (1.0 - x - y + 2.0 * z) + rho * (x + y - 2.0 * z) + dzdrho = (x - z) * (y - z) / denom + return math.log2((1.0 - z) / z) * dzdrho * rho * math.log(2.0) + + +# ====================================================================== +# the margin-modulated census (float path) +# ====================================================================== + +def census_mm(u: dict[int, float], n: int, lam: float) -> dict: + """Per-history margin-modulated bookkeeping over i < n. + + Returns the three MM aggregates plus the plain aggregate recomputed from + the same tables (anchor against agg_eval), max marginal, entropy, + dichotomy flag, and the per-i extreme MM contributions. + """ + pi = ORA.tilt_of_potential(u, lam) + supp = {a for (a, _b) in pi} + rho_t = 2.0 ** lam + + num_sec = 0.0 # sum m * (h2(z~) - h2(z_target)) + num_der = 0.0 # sum m * dh_dlam * (l2or - lam) + num_abs = 0.0 # sum m * |dh_dlam| * (l2or - lam) + den_mass = 0.0 + den_absw = 0.0 + num_plain = 0.0 + per_i_sec: dict[int, float] = {} + dich_all = True + n_rows = 0 + + for i in range(1, n): # i < n only + pmask = (1 << (i - 1)) - 1 + bit = 1 << (i - 1) + prefixes = {A & pmask for A in supp} + active = {c for c in prefixes + if any(A & pmask == c and A & bit for A in supp) + and any(A & pmask == c and not A & bit for A in supp)} + tables: dict[tuple[int, int], list[float]] = {} + for (A, B), w in pi.items(): + key = (A & pmask, B & pmask) + tab = tables.setdefault(key, [0.0] * 4) + tab[(2 if A & bit else 0) + (1 if B & bit else 0)] += w + i_sec = 0.0 + for (a, b), (p00, p01, p10, p11) in tables.items(): + nondeg = min(p00, p01, p10, p11) > 0.0 + if nondeg != (a in active and b in active): + dich_all = False + if not nondeg: + continue + mass = p00 + p01 + p10 + p11 + l2or = (math.log2(p11) + math.log2(p00) + - math.log2(p10) - math.log2(p01)) + x = (p00 + p01) / mass # P(A_i = 0 | hist) + y = (p00 + p10) / mass # P(B_i = 0 | hist) + z_rlz = p00 / mass + z_t = z_plackett(x, y, rho_t) + w_der = dh_dlam(x, y, rho_t) + d = l2or - lam + sec = mass * (h2(z_rlz) - h2(z_t)) + num_sec += sec + i_sec += sec + num_der += mass * w_der * d + num_abs += mass * abs(w_der) * d + den_mass += mass + den_absw += mass * abs(w_der) + num_plain += mass * d + n_rows += 1 + per_i_sec[i] = i_sec + + if den_mass == 0.0: + return None + mu = ORA.mu_of(pi) + worst_i = min(per_i_sec, key=lambda k: per_i_sec[k]) + return { + "mm_sec": num_sec / den_mass, + "mm_der": num_der / (den_absw if den_absw > 0 else 1.0), + "mm_abs": num_abs / (den_absw if den_absw > 0 else 1.0), + "agg_plain": num_plain / den_mass, + "num_sec": num_sec, "num_der": num_der, "num_abs": num_abs, + "den_mass": den_mass, + "max_marginal": max(ORA.marginal_elements(mu, n)), + "H": ORA.entropy_of(mu), + "dichotomy_ok": dich_all, "n_rows": n_rows, + "per_i_sec_min": per_i_sec[worst_i], "per_i_sec_argmin": worst_i, + } + + +# ====================================================================== +# part A -- anchors +# ====================================================================== + +def part_a() -> dict: + out = {} + wit = ORA.WITNESS_U + + # 1. plain aggregate reproduces 013/016 + ev = agg_eval(wit, 7, 3.5) + log(f"[A] witness plain aggregate (lam 3.5): {ev['agg']:+.6f} " + f"(013 certified +1.844669)") + out["witness_agg"] = ev["agg"] + ok_wit = abs(ev["agg"] - 1.844669) < 1e-3 + + # 2. census_mm's internal plain aggregate agrees with agg_eval + mm = census_mm(wit, 7, 3.5) + log(f"[A] census_mm plain-agg cross-check: {mm['agg_plain']:+.6f} " + f"(diff {abs(mm['agg_plain'] - ev['agg']):.2e})") + ok_cross = abs(mm["agg_plain"] - ev["agg"]) < 1e-9 + + # 3. 016's theta* kill at n = 96, lam = 2 reproduces + th = dict(THETA_STAR_L2) + th = tune_dil(96, 90, 2.0, th) + ev96 = agg_eval(mu_ladder_theta(96, 90, th), 96, 2.0) + log(f"[A] theta* ladder n=96 lam=2 plain aggregate: {ev96['agg']:+.9f} " + f"(016 certified -0.000759865)") + ok_kill = ev96["agg"] < 0 and abs(ev96["agg"] + 0.000759865) < 5e-5 + + # 4. Plackett round-trip on real census rows: z_plackett(x, y, OR) + # must equal the realized z~ (the 2x2 table is determined by its + # margins and OR) + pi = ORA.tilt_of_potential(wit, 3.5) + supp = {a for (a, _b) in pi} + worst = 0.0 + n_checked = 0 + for i in range(1, 7): + pmask, bit = (1 << (i - 1)) - 1, 1 << (i - 1) + tables: dict[tuple[int, int], list[float]] = {} + for (A, B), w in pi.items(): + key = (A & pmask, B & pmask) + tab = tables.setdefault(key, [0.0] * 4) + tab[(2 if A & bit else 0) + (1 if B & bit else 0)] += w + for (p00, p01, p10, p11) in tables.values(): + if min(p00, p01, p10, p11) <= 0: + continue + mass = p00 + p01 + p10 + p11 + x, y, z = (p00 + p01) / mass, (p00 + p10) / mass, p00 / mass + orr = (p00 * p11) / (p01 * p10) + worst = max(worst, abs(z_plackett(x, y, orr) - z)) + n_checked += 1 + log(f"[A] Plackett round-trip on {n_checked} witness rows: " + f"max |z_rho(x,y,OR) - z~| = {worst:.2e}") + ok_rt = worst < 1e-12 + + # 5. secant-vs-derivative consistency: on one row-like configuration, + # h2(z_rho2) - h2(z_rho1) ~ integral of dh_dlam + x, y = 0.62, 0.55 + l1, l2 = 1.0, 1.001 + sec = h2(z_plackett(x, y, 2 ** l2)) - h2(z_plackett(x, y, 2 ** l1)) + der = dh_dlam(x, y, 2 ** l1) * (l2 - l1) + log(f"[A] secant vs derivative (dl=1e-3): {sec:.9f} vs {der:.9f} " + f"(rel diff {abs(sec - der) / abs(der):.1e})") + ok_sd = abs(sec - der) / abs(der) < 1e-2 + + # 6. MM padding invariance: MU(20,1) == witness on the MM aggregates + mm20 = census_mm(mu_ladder(20, 1), 20, 3.5) + d_pad = abs(mm20["mm_sec"] - mm["mm_sec"]) + log(f"[A] MM padding invariance MU(20,1) vs witness: diff {d_pad:.2e}") + ok_pad = d_pad < 1e-10 + + out.update({"ok_witness": ok_wit, "ok_cross": ok_cross, + "ok_kill_repro": ok_kill, "kill_repro_agg": ev96["agg"], + "ok_roundtrip": ok_rt, "ok_secant_deriv": ok_sd, + "ok_padding": ok_pad, "witness_mm_sec": mm["mm_sec"], + "witness_mm_der": mm["mm_der"], "witness_mm_abs": mm["mm_abs"]}) + ok = all(v for k, v in out.items() if k.startswith("ok_")) + log(f"[A] anchors {'ALL PASS' if ok else '*** FAILURE ***'}") + out["all_ok"] = ok + save("gap1c_partA.json", out) + if not ok: + raise SystemExit("anchor failure -- do not trust anything downstream") + return out + + +# ====================================================================== +# part B -- MM on witness and ladders +# ====================================================================== + +def theta_ladder_at(n: int, lam: float, th_base: dict) -> dict[int, float]: + r = n - 6 + th = tune_dil(n, r, lam, th_base) + return mu_ladder_theta(n, r, th) + + +def part_b() -> dict: + out_rows = [] + wit = ORA.WITNESS_U + + mmw = census_mm(wit, 7, 3.5) + log(f"[B] witness (n=7, lam=3.5): plain {mmw['agg_plain']:+.4f} " + f"MM_sec {mmw['mm_sec']:+.6f} MM_der {mmw['mm_der']:+.6f} " + f"MM_abs {mmw['mm_abs']:+.6f}") + out_rows.append({"name": "witness", "n": 7, "lambda": 3.5, **{ + k: mmw[k] for k in ("mm_sec", "mm_der", "mm_abs", "agg_plain", + "max_marginal", "dichotomy_ok")}}) + + # ladders: raw witness weights and theta*(lam=2), lam = 2 + ns = (16, 32, 48, 64, 96, 128, 160) if not FAST else (16, 32) + for name, th_base in (("raw", THETA_WITNESS), ("theta*", THETA_STAR_L2)): + for n in ns: + u = theta_ladder_at(n, 2.0, th_base) + mm = census_mm(u, n, 2.0) + log(f"[B] {name:7s} ladder n={n:3d} lam=2: " + f"plain {mm['agg_plain']:+.6f} MM_sec {mm['mm_sec']:+.6e} " + f"MM_der {mm['mm_der']:+.6e} MM_abs {mm['mm_abs']:+.6e} " + f"(mm {mm['max_marginal']:.3f})") + out_rows.append({"name": f"{name}_ladder", "n": n, "lambda": 2.0, + **{k: mm[k] for k in + ("mm_sec", "mm_der", "mm_abs", "agg_plain", + "max_marginal", "dichotomy_ok", + "per_i_sec_min", "per_i_sec_argmin")}}) + + # lambda profile of the theta* ladder at n = 96 + if not FAST: + for lam in (0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5): + u = theta_ladder_at(96, lam, THETA_STAR_L2) + mm = census_mm(u, 96, lam) + log(f"[B] theta* ladder n=96 lam={lam:3.1f}: " + f"plain {mm['agg_plain']:+.6f} MM_sec {mm['mm_sec']:+.6e} " + f"MM_abs {mm['mm_abs']:+.6e}") + out_rows.append({"name": "theta*_lamprof", "n": 96, "lambda": lam, + **{k: mm[k] for k in + ("mm_sec", "mm_der", "mm_abs", "agg_plain", + "max_marginal", "dichotomy_ok")}}) + + save("gap1c_partB.json", {"rows": out_rows}) + return {"rows": out_rows} + + +# ====================================================================== +# part C -- adversarial theta re-optimization against MM +# ====================================================================== + +def mm_objective(th: dict, n: int, r: int, lam: float, key: str = "mm_abs"): + """Objective = the |sigma|-weighted variant: parts A/B showed the two + SIGNED variants are already negative on the witness itself, so mm_abs is + the only form left standing as a candidate control -- it is what the + adversary must now attack.""" + u = mu_ladder_theta(n, r, th) + mm = census_mm(u, n, lam) + if mm is None: + return float("inf"), None + pen = 300.0 * max(0.0, mm["max_marginal"] - MARG_TARGET) + return mm[key] + pen, mm + + +def part_c() -> dict: + out_rows = [] + n_opt = 48 if not FAST else 16 + r_opt = n_opt - 6 + steps = 260 if not FAST else 30 + best_by_lam = {} + for lam in (2.0, 3.5): + rng = random.Random(180000 + int(lam * 10)) + th = dict(tune_dil(n_opt, r_opt, lam, THETA_STAR_L2)) + cur, mm = mm_objective(th, n_opt, r_opt, lam) + for step in range(steps): + sigma = 0.8 * (1 - step / steps) + 0.05 + cand = dict(th) + k = rng.choice(list(cand)) + cand[k] = max(cand[k], 1e-8) * math.exp(rng.gauss(0, sigma)) + v2, mm2 = mm_objective(cand, n_opt, r_opt, lam) + if v2 < cur: + th, cur, mm = cand, v2, mm2 + log(f"[C] MM theta-opt n={n_opt} lam={lam}: " + f"MM_abs {mm['mm_abs']:+.6e} MM_sec {mm['mm_sec']:+.6e} " + f"plain {mm['agg_plain']:+.4f} (mm {mm['max_marginal']:.3f})") + best_by_lam[lam] = dict(th) + out_rows.append({"stage": "opt", "n": n_opt, "lambda": lam, + "theta": th, **{k: mm[k] for k in + ("mm_sec", "mm_der", "mm_abs", + "agg_plain", "max_marginal")}}) + for n in ((64, 96, 128, 160) if not FAST else (24,)): + r = n - 6 + th_n = tune_dil(n, r, lam, th) + mm_n = census_mm(mu_ladder_theta(n, r, th_n), n, lam) + log(f"[C] MM theta({lam}) at n={n}: " + f"MM_abs {mm_n['mm_abs']:+.6e} MM_sec {mm_n['mm_sec']:+.6e} " + f"plain {mm_n['agg_plain']:+.4f} " + f"(mm {mm_n['max_marginal']:.3f})") + out_rows.append({"stage": "transfer", "n": n, "lambda": lam, + "theta": th_n, **{k: mm_n[k] for k in + ("mm_sec", "mm_der", "mm_abs", + "agg_plain", "max_marginal")}}) + + # free-support climbs against MM_abs at n = 8 (the ladder hides its + # deficit at near-degenerate margins, which |sigma| suppresses -- a + # violation of mm_abs needs deficits at HEALTHY margins, a geometry the + # theta-ladder cannot express; free atoms can) + rng = random.Random(181111) + best_free = None + wit8 = {a: w for a, w in ORA.WITNESS_U.items()} # n=7 embeds in n=8 + + def free_obj(m): + return (m["mm_abs"] + + 300.0 * max(0.0, m["max_marginal"] - MARG_TARGET)) + + for trial in range(8 if not FAST else 2): + lam = (1.0, 2.0, 3.5, 2.0, 3.5, 1.0, 2.0, 3.5)[trial % 8] + if trial < 3: + u = dict(wit8) + else: + u = {a: math.exp(rng.uniform(-1.5, 1.5)) + for a in rng.sample(range(256), 10)} + mm = census_mm(u, 8, lam) + if mm is None: + continue + cur = free_obj(mm) + for step in range(600 if not FAST else 30): + cand = dict(u) + a = rng.choice(list(cand)) + if rng.random() < 0.15 and len(cand) < 14: + cand[rng.getrandbits(8)] = math.exp(rng.uniform(-3, 1)) + else: + cand[a] = max(cand[a] * math.exp(rng.gauss(0, 0.5)), 1e-9) + mm2 = census_mm(cand, 8, lam) + if mm2 is None: + continue + v2 = free_obj(mm2) + if v2 < cur: + u, cur, mm = cand, v2, mm2 + row = {"trial": trial, "lambda": lam, "mm_abs": mm["mm_abs"], + "mm_sec": mm["mm_sec"], "max_marginal": mm["max_marginal"], + "u": {format(a, "08b"): w for a, w in u.items()}} + if (mm["max_marginal"] <= MARG_TARGET + and (best_free is None or mm["mm_abs"] < best_free["mm_abs"])): + best_free = row + log(f"[C] free climb n=8 lam={lam}: endpoint MM_abs " + f"{mm['mm_abs']:+.6e} MM_sec {mm['mm_sec']:+.6e} " + f"(mm {mm['max_marginal']:.3f})") + out = {"rows": out_rows, "best_free": best_free, + "best_theta_by_lam": {str(k): v for k, v in best_by_lam.items()}} + save("gap1c_partC.json", out) + return out + + +# ====================================================================== +# part D -- the lambda-window-restricted candidate +# ====================================================================== + +def part_d() -> dict: + out_rows = [] + + # window scan with theta* and raw weights, lam = LAMWIN(n) etc. + ns = (48, 96, 160, 224, 320) if not FAST else (24, 48) + for name, th_base in (("theta*", THETA_STAR_L2), ("raw", THETA_WITNESS)): + for n in ns: + lw = lamwin(n) + row = {"name": f"{name}_window", "n": n, "lamwin": lw, "points": []} + lams = [lw, lw / 2] + ([lw / 4] if n < 224 else []) + for lam in lams: + u = theta_ladder_at(n, lam, th_base) + ev = agg_eval(u, n, lam) + row["points"].append({"lambda": lam, "agg": ev["agg"], + "max_marginal": ev["max_marginal"], + "per_i_min": ev["per_i_min"]}) + # small-lambda fit A ~ a1 lam + a2 lam^2 from the two smallest + # lambda points; with 3 points the lw row is an out-of-fit check + p0 = row["points"][0] + p1, p2 = row["points"][-1], row["points"][-2] + l1, l2 = p1["lambda"], p2["lambda"] + a1 = (p1["agg"] * l2 * l2 - p2["agg"] * l1 * l1) / \ + (l1 * l2 * (l2 - l1)) + a2 = (p2["agg"] * l1 - p1["agg"] * l2) / (l1 * l2 * (l2 - l1)) + pred = a1 * p0["lambda"] + a2 * p0["lambda"] ** 2 + row.update({"a1": a1, "a2": a2, "fit_check_at_lw": + {"pred": pred, "actual": p0["agg"], + "in_fit": len(lams) == 2}}) + log(f"[D] {name:7s} n={n:3d} window lam={lw:.4f}: " + f"A(lw) {p0['agg']:+.6e} A(lw/2) {row['points'][1]['agg']:+.6e} " + f"a1 {a1:+.5e} a2 {a2:+.5e} a1*n {a1 * n:+.3f}") + out_rows.append(row) + + # theta re-optimization AT window lambda (the adversary adapts to the + # window), then per-n transfer where each n uses its own LAMWIN(n) + best_theta = None + for n_opt in ((48, 96) if not FAST else (24,)): + r_opt = n_opt - 6 + lam = lamwin(n_opt) + rng = random.Random(190000 + n_opt) + th = dict(tune_dil(n_opt, r_opt, lam, THETA_STAR_L2)) + ev = agg_eval(mu_ladder_theta(n_opt, r_opt, th), n_opt, lam) + cur = ev["agg"] + 100.0 * max(0.0, ev["max_marginal"] - MARG_TARGET) + steps = 300 if not FAST else 30 + for step in range(steps): + sigma = 0.8 * (1 - step / steps) + 0.05 + cand = dict(th) + k = rng.choice(list(cand)) + cand[k] = max(cand[k], 1e-8) * math.exp(rng.gauss(0, sigma)) + ev2 = agg_eval(mu_ladder_theta(n_opt, r_opt, cand), n_opt, lam) + if ev2 is None: + continue + v2 = (ev2["agg"] + + 100.0 * max(0.0, ev2["max_marginal"] - MARG_TARGET)) + if v2 < cur: + th, cur, ev = cand, v2, ev2 + log(f"[D] window theta-opt n={n_opt} lam={lam:.4f}: " + f"agg {ev['agg']:+.6e} (mm {ev['max_marginal']:.3f})") + out_rows.append({"stage": "window_opt", "n": n_opt, "lambda": lam, + "theta": th, "agg": ev["agg"], + "max_marginal": ev["max_marginal"]}) + if best_theta is None or ev["agg"] < best_theta[1]: + best_theta = (dict(th), ev["agg"], n_opt) + for n in ((96, 160, 224, 320) if not FAST else (48,)): + if n == n_opt: + continue + r = n - 6 + lw = lamwin(n) + th_n = tune_dil(n, r, lw, th) + ev_n = agg_eval(mu_ladder_theta(n, r, th_n), n, lw) + log(f"[D] window theta(n={n_opt}) at n={n} lam={lw:.4f}: " + f"agg {ev_n['agg']:+.6e} (mm {ev_n['max_marginal']:.3f})") + out_rows.append({"stage": "window_transfer", "n": n, "lambda": lw, + "from_n": n_opt, "theta": th_n, + "agg": ev_n["agg"], + "max_marginal": ev_n["max_marginal"]}) + + # free (theta, lam <= LAMWIN) climb: adversary also picks lambda inside + # the window + n_free = 96 if not FAST else 24 + r_free = n_free - 6 + lw = lamwin(n_free) + rng = random.Random(191111) + th = dict(tune_dil(n_free, r_free, lw, THETA_STAR_L2)) + lam = lw + ev = agg_eval(mu_ladder_theta(n_free, r_free, th), n_free, lam) + cur = ev["agg"] + 100.0 * max(0.0, ev["max_marginal"] - MARG_TARGET) + for step in range(200 if not FAST else 20): + sigma = 0.8 * (1 - step / 200) + 0.05 + cand, cl = dict(th), lam + if rng.random() < 0.25: + cl = min(lw, max(lw / 64, lam * math.exp(rng.gauss(0, sigma)))) + else: + k = rng.choice(list(cand)) + cand[k] = max(cand[k], 1e-8) * math.exp(rng.gauss(0, sigma)) + ev2 = agg_eval(mu_ladder_theta(n_free, r_free, cand), n_free, cl) + if ev2 is None: + continue + v2 = ev2["agg"] + 100.0 * max(0.0, ev2["max_marginal"] - MARG_TARGET) + if v2 < cur: + th, lam, cur, ev = cand, cl, v2, ev2 + log(f"[D] free (theta, lam<=window) climb n={n_free}: endpoint " + f"agg {ev['agg']:+.6e} at lam={lam:.4f} (window {lw:.4f})") + out_rows.append({"stage": "free_window_climb", "n": n_free, + "lambda": lam, "lamwin": lw, "theta": th, + "agg": ev["agg"], "max_marginal": ev["max_marginal"]}) + + out = {"rows": out_rows} + save("gap1c_partD.json", out) + return out + + +# ====================================================================== +# part E (--ext) -- raw-weight ladder extension (016 lead 3) +# ====================================================================== + +def part_e() -> dict: + out_rows = [] + for n in (160, 192, 224, 256, 288, 320): + r = n - 6 + best = None + for k in range(6): # 6-point dilution sweep + th = dict(THETA_WITNESS) + th["dil"] = 10.0 * (2.0 ** k) + marg = marginals_of_u(mu_ladder_theta(n, r, th), n, 2.0) + if max(marg) > MARG_TARGET: + continue + ev = agg_eval(mu_ladder_theta(n, r, th), n, 2.0) + if best is None or ev["agg"] < best["agg"]: + best = {"agg": ev["agg"], "dil": th["dil"], + "max_marginal": ev["max_marginal"]} + th = tune_dil(n, r, 2.0) # plus the bisection tuner + ev = agg_eval(mu_ladder_theta(n, r, th), n, 2.0) + if best is None or ev["agg"] < best["agg"]: + best = {"agg": ev["agg"], "dil": th["dil"], + "max_marginal": ev["max_marginal"]} + log(f"[E] raw ladder n={n}: honest min agg {best['agg']:+.6f} " + f"(dil {best['dil']:.1f}, mm {best['max_marginal']:.3f})") + out_rows.append({"n": n, **best}) + save("gap1c_partE.json", {"rows": out_rows}) + return {"rows": out_rows} + + +# ====================================================================== +# part F (--cert) -- exact certification +# ====================================================================== + +F0, F1 = Fraction(0), Fraction(1) + + +def h2_enclosure(z: Fraction) -> tuple[Fraction, Fraction]: + """[lo, hi] for h2(z), z rational in (0, 1).""" + if z <= 0 or z >= 1: + return F0, F0 + l1lo, l1hi = ORS.log2_enclosure(z) + l2lo, l2hi = ORS.log2_enclosure(1 - z) + return (-z * l1hi - (1 - z) * l2hi), (-z * l1lo - (1 - z) * l2lo) + + +def z_target_enclosure(x: Fraction, y: Fraction, t: Fraction, + steps: int = 80) -> tuple[Fraction, Fraction]: + """Dyadic bisection enclosure of the Plackett root z_t(x, y): + F(z) = z(1-x-y+z) - t(x-z)(y-z), unique root in the coupling range. + For t > 1 the root lies in (xy, min(x,y)); t < 1 in (max(0,x+y-1), xy).""" + if t == 1: + return x * y, x * y + + def bigF(z: Fraction) -> Fraction: + return z * (1 - x - y + z) - t * (x - z) * (y - z) + + if t > 1: + lo, hi = x * y, min(x, y) + else: + lo, hi = max(F0, x + y - 1), x * y + # F(lo) < 0 < F(hi) for t > 1 (and reversed signs handled uniformly) + flo = bigF(lo) + for _ in range(steps): + mid = (lo + hi) / 2 + # keep denominators dyadic-bounded + mid = Fraction(mid.numerator, mid.denominator) + fm = bigF(mid) + if fm == 0: + return mid, mid + if (fm < 0) == (flo < 0): + lo, flo = mid, fm + else: + hi = mid + return lo, hi + + +def h2_over_interval(zlo: Fraction, zhi: Fraction) -> tuple[Fraction, Fraction]: + """[lo, hi] of h2 over [zlo, zhi] (h2 unimodal, peak 1 at z = 1/2).""" + alo, ahi = h2_enclosure(zlo) + blo, bhi = h2_enclosure(zhi) + lo = min(alo, blo) + hi = max(ahi, bhi) + if zlo < Fraction(1, 2) < zhi: + hi = F1 + return lo, hi + + +def exact_mm(u_float: dict[int, float], n: int, t: Fraction, + rationalize=None): + """Certified enclosure of the MM_sec numerator (and the aggregate) at + rational tilt t for the exact measure whose potential is the + rationalization of u_float. Caches by (x, y) -- ladder units are + exchangeable so the distinct-margin count is tiny compared to rows.""" + if rationalize is None: + u = {a: Fraction(w) for a, w in u_float.items()} + else: + u = {a: rationalize(w) for a, w in u_float.items()} + u = {a: w for a, w in u.items() if w > 0} + cens = ORS.exact_census(u, t, n) + zcache: dict[tuple[Fraction, Fraction], tuple] = {} + num_lo = num_hi = F0 + plain_lo = plain_hi = F0 + wtot = F0 + dich = True + n_rows = 0 + for c in cens: + if c["i"] >= n: + continue + dich = dich and c["dich_ok"] + active = c["active"] + for (a, b), tab in c["tables"].items(): + p00, p01, p10, p11 = tab + if min(p00, p01, p10, p11) <= 0: + continue + assert a in active and b in active + mass = p00 + p01 + p10 + p11 + x, y = (p00 + p01) / mass, (p00 + p10) / mass + z_rlz = p00 / mass + key = (x, y) + if key not in zcache: + zlo, zhi = z_target_enclosure(x, y, t) + zcache[key] = h2_over_interval(zlo, zhi) + htlo, hthi = zcache[key] + hrlo, hrhi = h2_enclosure(z_rlz) + num_lo += mass * (hrlo - hthi) + num_hi += mass * (hrhi - htlo) + orr = (p11 * p00) / (p10 * p01) + llo, lhi = ORS.log2_enclosure(orr / t) + plain_lo += mass * llo + plain_hi += mass * lhi + wtot += mass + n_rows += 1 + _mu, marg = ORS.exact_marginals(u, t, n) + return {"mm_num_lo": num_lo, "mm_num_hi": num_hi, + "mm_lo": num_lo / wtot, "mm_hi": num_hi / wtot, + "agg_lo": plain_lo / wtot, "agg_hi": plain_hi / wtot, + "max_marg": max(marg), "in_regime": max(marg) < ORS.RECORD, + "dich_ok": dich, "n_rows": n_rows, + "n_distinct_margins": len(zcache)} + + +def tidy(w: float) -> Fraction: + return Fraction(w).limit_denominator(10 ** 4) + + +def part_f() -> dict: + """Job list is fixed from the B-D probe outcomes (see the attempt + record): certify the decisive MM and window points.""" + out_rows = [] + jobs = [] + + # (1) MM on the witness (007 lead 2's first ask, now certified) + jobs.append(("mm_witness_t181/16", ORA.WITNESS_U, 7, + Fraction(181, 16), "mm", None)) + + # (2) MM on the theta* kill instance: does the margin-modulated + # aggregate survive on the exact family that kills the plain one? + th = tune_dil(96, 90, 2.0, dict(THETA_STAR_L2)) + u96 = mu_ladder_theta(96, 90, th) + jobs.append(("mm_theta*_n96_t4", u96, 96, Fraction(4), "mm", None)) + + # (3) same at n = 160 + th160 = tune_dil(160, 154, 2.0, dict(THETA_STAR_L2)) + u160 = mu_ladder_theta(160, 154, th160) + jobs.append(("mm_theta*_n160_t4", u160, 160, Fraction(4), "mm", tidy)) + + # (4) window points: plain aggregate at a rational tilt inside the + # window at n = 96 and 160 (LAMWIN(96) ~ 0.0521, log2(26/25) ~ + # 0.0566 > it, log2(51/50) ~ 0.0286 < it -> use 51/50 at 96; + # LAMWIN(160) ~ 0.0309 -> 51/50 works there too) + for n, u in ((96, None), (160, None)): + lw = lamwin(n) + t_win = Fraction(51, 50) + lam_t = math.log2(51 / 50) + assert lam_t < lw, (n, lam_t, lw) + r = n - 6 + thw = tune_dil(n, r, lam_t, dict(THETA_STAR_L2)) + uw = mu_ladder_theta(n, r, thw) + jobs.append((f"window_theta*_n{n}_t51/50", uw, n, t_win, "plain", + tidy if n >= 160 else None)) + + for name, u, n, t, kind, rat in jobs: + t_start = time.monotonic() + if kind == "mm": + r = exact_mm(u, n, t, rationalize=rat) + lo, hi = float(r["mm_lo"]), float(r["mm_hi"]) + verdict = ("CERTIFIED NEGATIVE (KILL)" if r["mm_hi"] < 0 else + "CERTIFIED POSITIVE" if r["mm_lo"] > 0 else + "STRADDLES 0") + log(f"[F] exact MM {name}: MM_sec in [{lo:+.9e}, {hi:+.9e}] " + f"{verdict}; plain in [{float(r['agg_lo']):+.6f}, " + f"{float(r['agg_hi']):+.6f}]; in-regime {r['in_regime']}; " + f"dich {r['dich_ok']}; rows {r['n_rows']} " + f"({r['n_distinct_margins']} distinct margins); " + f"{time.monotonic() - t_start:.0f}s") + out_rows.append({ + "name": name, "n": n, "t": str(t), "kind": kind, + "mm_lo": dec_bound(r["mm_lo"]), + "mm_hi": dec_bound(r["mm_hi"], up=True), + "mm_lo_float": lo, "mm_hi_float": hi, + "agg_lo_float": float(r["agg_lo"]), + "agg_hi_float": float(r["agg_hi"]), + "in_regime": r["in_regime"], + "max_marg_float": float(r["max_marg"]), + "dich_ok": r["dich_ok"], "verdict": verdict}) + else: + u_in = ({a: float(tidy(w)) for a, w in u.items()} + if rat is not None else u) + r = exact_aggregate(u_in, n, t) + lo, hi = float(r["agg_lo"]), float(r["agg_hi"]) + verdict = ("CERTIFIED NEGATIVE (KILL)" if r["agg_hi"] < 0 else + "CERTIFIED POSITIVE" if r["agg_lo"] > 0 else + "STRADDLES 0") + log(f"[F] exact plain {name}: agg in [{lo:+.9e}, {hi:+.9e}] " + f"{verdict}; in-regime {r['in_regime']}; dich {r['dich_ok']}; " + f"{time.monotonic() - t_start:.0f}s") + out_rows.append({ + "name": name, "n": n, "t": str(t), "kind": kind, + "agg_lo": dec_bound(r["agg_lo"]), + "agg_hi": dec_bound(r["agg_hi"], up=True), + "agg_lo_float": lo, "agg_hi_float": hi, + "in_regime": r["in_regime"], + "max_marg_float": float(r["max_marg"]), + "dich_ok": r["dich_ok"], "verdict": verdict}) + save("gap1c_partF.json", {"rows": out_rows}) + return {"rows": out_rows} + + +# ====================================================================== +# part R (--robust) -- robustness battery for the witness MM_sec kill, +# plus MM on friendly genres (is the negativity generic or structural?) +# ====================================================================== + +def part_r() -> dict: + out = {} + wit = ORA.WITNESS_U + + # product anchors: MM_sec must vanish identically (OR = 2^lam at every + # history, so z~ = z_target row by row) + prows = [] + for n, p, lam in ((6, 0.3, 2.0), (8, 0.38, 1.0)): + u = {a: (p / (1 - p)) ** ORA.popcount(a) for a in range(1 << n)} + mm = census_mm(u, n, lam) + prows.append({"n": n, "p": p, "lambda": lam, "mm_sec": mm["mm_sec"]}) + log(f"[R] product Bern({p})^{n} lam={lam}: MM_sec {mm['mm_sec']:+.1e}") + out["product_anchors"] = prows + + # friendly non-product genres: MM_sec sign is NOT generically negative + frows = [] + n = 8 + genres = {} + eps = 0.04 + genres["d0_bern"] = {0: 1.0} | { + a: 0.55 * (0.5 + eps) ** ORA.popcount(a) + * (0.5 - eps) ** (n - ORA.popcount(a)) for a in range(1, 1 << n)} + genres["bern_mix"] = { + a: 0.6 * 0.2 ** ORA.popcount(a) * 0.8 ** (n - ORA.popcount(a)) + + 0.4 * 0.45 ** ORA.popcount(a) * 0.55 ** (n - ORA.popcount(a)) + for a in range(1 << n)} + genres["smoothed_slice"] = { + a: (1.0 if ORA.popcount(a) == 3 else 1e-3) for a in range(1 << n)} + rng = random.Random(7) + for tr in range(3): + genres[f"random_dense_{tr}"] = { + a: math.exp(rng.gauss(0, 0.35)) for a in range(1 << 6)} + for name, u in genres.items(): + nn = 6 if name.startswith("random") else n + for lam in (1.0, 2.0): + mm = census_mm(u, nn, lam) + frows.append({"name": name, "n": nn, "lambda": lam, + **{k: mm[k] for k in ("mm_sec", "mm_der", "mm_abs", + "agg_plain", "max_marginal")}}) + log(f"[R] {name:16s} lam={lam}: MM_sec {mm['mm_sec']:+.5f} " + f"MM_der {mm['mm_der']:+.5f} (mm {mm['max_marginal']:.3f})") + out["friendly_genres"] = frows + + # witness kill robustness: 20 x 3% perturbations, 2-digit tidy, lambda + # profile + rng = random.Random(2024) + pert = [] + for _ in range(20): + u = {a: w * math.exp(rng.uniform(-0.03, 0.03)) + for a, w in wit.items()} + mm = census_mm(u, 7, 3.5) + pert.append({"mm_sec": mm["mm_sec"], + "max_marginal": mm["max_marginal"]}) + n_fail = sum(1 for r in pert + if r["mm_sec"] >= 0 or r["max_marginal"] > 0.38271) + log(f"[R] witness 20 x 3% perturbations: {n_fail} failures; MM_sec in " + f"[{min(r['mm_sec'] for r in pert):+.5f}, " + f"{max(r['mm_sec'] for r in pert):+.5f}]") + out["perturbations"] = {"rows": pert, "n_fail": n_fail} + + u2 = {a: float(f"{w:.2g}") for a, w in wit.items()} + mm2 = census_mm(u2, 7, 3.5) + out["tidy_2digit"] = {"mm_sec": mm2["mm_sec"], + "max_marginal": mm2["max_marginal"]} + log(f"[R] 2-digit tidy witness: MM_sec {mm2['mm_sec']:+.5f} " + f"(mm {mm2['max_marginal']:.4f})") + + prof = [] + for l in (0.25, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 5.0): + mm = census_mm(wit, 7, l) + prof.append({"lambda": l, **{k: mm[k] for k in + ("mm_sec", "mm_der", "mm_abs")}}) + log(f"[R] witness lam={l:4.2f}: MM_sec {mm['mm_sec']:+.6f} " + f"MM_der {mm['mm_der']:+.6f} MM_abs {mm['mm_abs']:+.6f}") + out["witness_lambda_profile"] = prof + save("gap1c_partR.json", out) + return out + + +def main() -> None: + if "--ext" in sys.argv: + part_e() + return + if "--cert" in sys.argv: + part_a() + part_f() + return + if "--robust" in sys.argv: + part_a() + part_r() + return + part_a() + part_b() + part_c() + part_d() + log("done (parts A-D)") + + +if __name__ == "__main__": + main() diff --git a/problems/union-closed/explore/uc_gap1c_skeptic.py b/problems/union-closed/explore/uc_gap1c_skeptic.py new file mode 100644 index 0000000..b346233 --- /dev/null +++ b/problems/union-closed/explore/uc_gap1c_skeptic.py @@ -0,0 +1,935 @@ +#!/usr/bin/env python3 +"""Attempt 019: skeptic re-verification of 018 (margin-modulated control +killed at the witness; MM_abs and the lambda-window control survive the +ladder). + +INDEPENDENCE CONTRACT. This file imports nothing from +uc_gap1_candidates.py, uc_or_avg.py, uc_or_avg_skeptic.py, +uc_agg_ctrl_probe.py or uc_agg_ctrl_probe2.py. Everything below is written +from the PROSE of 007 section 1 (the well-posed M_i census), 018's record +(the three MM readings), and 016 section 1 (the MU(n, r) ladder). Witness +atoms, theta* weights and search endpoints are DATA copied from records / +checkpoints -- data reuse is fine, code reuse is not. + +The exact kit is new and structurally different from BOTH prior kits: + - log2 enclosure by dyadic interval squaring (digit extraction) with + directed rounding -- the idea 017 used, re-implemented here from + scratch; NOT 013's atanh-series kit (which 018 imported). + - Plackett-root enclosure by exact rational NEWTON steps with an exact + sign-checked bracket (every bracket update verified by an exact sign + evaluation of the quadratic; bisection fallback guarantees progress); + NOT 018's blind 80-step dyadic bisection. Bracket endpoint signs are + also proved symbolically: with F(z) = z(1-x-y+z) - t(x-z)(y-z) and + 0 < x, y < 1, t > 1: + F(xy) = (1-t) xy (1-x)(1-y) < 0, + F(min(x,y)) = min(x,y) (1 - max(x,y)) > 0, + so the root bracket (xy, min(x,y)) is certified before any iteration. + - the census is scaled-integer (exact) end-to-end; the only outward + rounding is inside the two enclosure primitives, which are directed. + +Parts: + S0 kit self-tests (log2 enclosure vs known values; Plackett root vs + closed forms and float solver; h2 enclosure; bracket sign identities). + S1 K1: from-scratch exact certification of MM_sec on the 10-atom witness + at t = 181/16 (plus the 2-digit tidy witness), with exact marginals, + exact dichotomy, and an exact per-row check that degenerate histories + contribute 0 to the secant numerator under either convention. + S2 K2: the definitional fork -- own float census evaluating 018's three + readings PLUS the variants 018 did not test (sensitivity at the + REALIZED odds ratio, signed and unsigned; |dh/drho| weighting). + S3 K3: float spot-checks with the ladder rebuilt from 016's prose spec + (own tune_dil): witness MM numbers, theta* ladder n = 96 at lam = 2, + raw ladder n = 96 (the sign 018's prose gets wrong), part-D window + value at n = 96 and the a1/a2 fit arithmetic, perturbation battery + with DIFFERENT seeds. + S4 K4: re-evaluation of the part-C best_free endpoint (data from + gap1c_partC.json) and the z = 1/2 sensitivity-boundary arithmetic + (the golden-ratio parenthetical). + +Standard library only. Deterministic (fixed seeds). Runtime ~3 min. +Usage: + python problems/union-closed/explore/uc_gap1c_skeptic.py [--fast] +Checkpoints: problems/union-closed/data/gap1csk_*.json +Log: tee to problems/union-closed/data/gap1csk_run.log +""" + +from __future__ import annotations + +import json +import math +import random +import sys +import time +from fractions import Fraction +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +DATA = ROOT / "data" + +RECORD_NUM, RECORD_DEN = 38271, 100000 # record threshold 0.38271 (Liu) +MARG_TARGET = 0.375 # 016's tuning target +F0, F1, HALF = Fraction(0), Fraction(1), Fraction(1, 2) +FAST = "--fast" in sys.argv +T0 = time.monotonic() + + +def log(msg: str) -> None: + print(f"[{time.monotonic() - T0:6.0f}s] {msg}", flush=True) + + +def save(name: str, obj) -> None: + DATA.mkdir(exist_ok=True) + (DATA / name).write_text(json.dumps(obj, indent=1, sort_keys=True, + default=str) + "\n", encoding="utf-8") + + +# ====================================================================== +# DATA (copied from records/checkpoints; not imported) +# ====================================================================== + +# 007's 10-atom witness (record part T / uc_or_avg.py WITNESS_U -- data) +WITNESS_U = { + 0b0100001: 3.802705744653898, # {1,6} + 0b1000001: 8.094727225702503, # {1,7} + 0b1010001: 0.004792968160237178, # {1,5,7} light slice + 0b0000000: 0.2526660836353666, # {} + 0b0100000: 27.28641199464732, # {6} + 0b1000000: 12.529780963546393, # {7} + 0b0110000: 0.3609234435898425, # {5,6} + 0b0000010: 20.0, # {2} dilution + 0b0000100: 20.0, # {3} dilution + 0b0001000: 20.0, # {4} dilution +} + +# theta* from 016 P2 (aggprobe2_partP2.json), as quoted in 018's engine +THETA_STAR_L2 = { + "A1": 14.026917090492047, "A2": 22.622718089566355, + "B0": 13.573329717299568, "B1": 43.00957928858568, + "B2": 24.62436309125563, "dil": 21.044172077650057, + "light": 0.007943082225456255, "resp": 0.0017259962695652103, +} + +# raw witness weights as a theta dict (016's parametrization -- data) +THETA_WITNESS = { + "A1": 3.802705744653898, "A2": 8.094727225702503, + "light": 0.004792968160237178, "B0": 0.2526660836353666, + "B1": 27.28641199464732, "B2": 12.529780963546393, + "resp": 0.3609234435898425, "dil": 20.0, +} + +LAMWIN_C = 4.847 # 009/011 window law c/(n-3) + + +# ====================================================================== +# certified log2 enclosure -- own implementation (interval squaring) +# ====================================================================== + +# ln 2 = -ln(1/2) = sum_{k>=1} (1/2)^k / k ; the partial sum after K terms +# is a strict lower bound and the tail is < 1/((K+1) 2^K). +_LN2_K = 72 +LN2_LO = sum(Fraction(1, k << k) for k in range(1, _LN2_K + 1)) +LN2_HI = LN2_LO + Fraction(1, (_LN2_K + 1) * (1 << _LN2_K)) + + +def log2_enc(q: Fraction, K: int = 96, prec: int = 288 + ) -> tuple[Fraction, Fraction]: + """Certified [lo, hi] with lo <= log2 q <= hi for rational q > 0. + + Write q = 2^e x, x in [1, 2). Square a directed-rounded dyadic + enclosure of x K times, renormalizing into [1, 4) by shifts whose count + builds the binary expansion S of 2^K log2 x; then + log2 x = (S + log2 x_K)/2^K with x_K in a known small interval, where + the elementary sandwich 1 - 1/v <= ln v <= v - 1 (v > 0) plus the LN2 + bounds give a crude certified enclosure whose error is divided by 2^K. + """ + num, den = q.numerator, q.denominator + if num <= 0: + raise ValueError("log2 of nonpositive") + if num & (num - 1) == 0 and den & (den - 1) == 0: # exact power of two + return (Fraction(num.bit_length() - den.bit_length()),) * 2 + e = num.bit_length() - den.bit_length() + nn, dd = (num, den << e) if e >= 0 else (num << (-e), den) + if nn < dd: + nn <<= 1 + e -= 1 + # x = nn/dd in [1, 2) + scale = 1 << prec + xlo = (nn << prec) // dd # floor + xhi = -((-(nn << prec)) // dd) # ceil + S = 0 + for _ in range(K): + xlo = (xlo * xlo) >> prec # round down + xhi = -((-(xhi * xhi)) >> prec) # round up + s = 0 + while xlo >= (scale << 1): # whole interval >= 2 + xlo >>= 1 + xhi = -((-xhi) >> 1) + s += 1 + while xhi < scale: # whole interval < 1 + xlo <<= 1 + xhi <<= 1 + s -= 1 + S = (S << 1) + s + vlo, vhi = Fraction(xlo, scale), Fraction(xhi, scale) + assert 0 < vlo <= vhi + ln_lo = 1 - 1 / vlo # <= ln vlo + ln_hi = vhi - 1 # >= ln vhi + l2_lo = ln_lo / (LN2_HI if ln_lo >= 0 else LN2_LO) + l2_hi = ln_hi / (LN2_LO if ln_hi >= 0 else LN2_HI) + pw = 1 << K + return e + (S + l2_lo) / pw, e + (S + l2_hi) / pw + + +def h2_enc(z: Fraction) -> tuple[Fraction, Fraction]: + """Certified [lo, hi] for binary entropy h2(z), z rational in [0, 1].""" + if z <= 0 or z >= 1: + return F0, F0 + alo, ahi = log2_enc(z) + blo, bhi = log2_enc(1 - z) + return (-z * ahi - (1 - z) * bhi), (-z * alo - (1 - z) * blo) + + +def h2_over(zlo: Fraction, zhi: Fraction) -> tuple[Fraction, Fraction]: + """[lo, hi] of h2 over [zlo, zhi] (h2 unimodal with peak 1 at 1/2).""" + alo, ahi = h2_enc(zlo) + blo, bhi = h2_enc(zhi) + lo, hi = min(alo, blo), max(ahi, bhi) + if zlo < HALF < zhi: + hi = F1 + return lo, hi + + +# ====================================================================== +# certified Plackett root -- exact Newton with sign-checked bracket +# ====================================================================== + +def plackett_F_coeffs(x: Fraction, y: Fraction, t: Fraction): + """F(z) = z(1-x-y+z) - t(x-z)(y-z) = A z^2 + B z + C.""" + return 1 - t, 1 - x - y + t * (x + y), -t * x * y + + +def z_root_enc(x: Fraction, y: Fraction, t: Fraction, + bits: int = 160) -> tuple[Fraction, Fraction]: + """Certified enclosure of the Plackett both-zero probability z_t(x, y) + for 0 < x, y < 1 and t > 1: the unique root of F in (xy, min(x,y)). + + Every bracket move is validated by an exact sign evaluation of F, so + correctness never depends on the Newton formula; Newton (on the exact + quadratic) supplies fast candidates, bisection guarantees progress. + """ + if t == 1: + return x * y, x * y + assert t > 1 and 0 < x < 1 and 0 < y < 1 + A, B, C = plackett_F_coeffs(x, y, t) + + def F(z: Fraction) -> Fraction: + return (A * z + B) * z + C + + lo, hi = x * y, min(x, y) + flo, fhi = F(lo), F(hi) + # symbolic identities, re-checked exactly per call + assert flo == (1 - t) * x * y * (1 - x) * (1 - y) and flo < 0 + assert fhi == min(x, y) * (1 - max(x, y)) and fhi > 0 + tol = Fraction(1, 1 << bits) + while hi - lo > tol: + probes = [(lo + hi) / 2] + d = 2 * A * hi + B + if d != 0: + zn = (hi - fhi / d).limit_denominator(1 << 400) + if lo < zn < hi: + probes.append(zn) + d = 2 * A * lo + B + if d != 0: + zn = (lo - flo / d).limit_denominator(1 << 400) + if lo < zn < hi: + probes.append(zn) + for z in probes: + fz = F(z) + if fz == 0: + return z, z + if fz < 0: + if z > lo: + lo, flo = z, fz + else: + if z < hi: + hi, fhi = z, fz + return lo, hi + + +# ====================================================================== +# exact census (scaled integer) and certified MM_sec / plain aggregate +# ====================================================================== + +def rationalize(u_float: dict[int, float]) -> dict[int, Fraction]: + return {a: Fraction(w) for a, w in u_float.items() if w > 0} + + +def exact_kernel(u: dict[int, Fraction], n: int, t: Fraction): + """Scaled-integer kernel k(A,B) = U_A U_B tn^p td^(n-p), p = |A&B|. + Global scale cancels in every normalized quantity.""" + supp = sorted(u) + L = 1 + for w in u.values(): + L = L * w.denominator // math.gcd(L, w.denominator) + U = {a: int(u[a] * L) for a in supp} + tn, td = t.numerator, t.denominator + ptn = [1] * (n + 1) + ptd = [1] * (n + 1) + for j in range(1, n + 1): + ptn[j] = ptn[j - 1] * tn + ptd[j] = ptd[j - 1] * td + rows = [] + for A in supp: + UA = U[A] + rows.append([UA * U[B] * ptn[bin(A & B).count("1")] + * ptd[n - bin(A & B).count("1")] for B in supp]) + return supp, rows + + +def exact_mm_cert(u_float: dict[int, float], n: int, t: Fraction) -> dict: + """Certified enclosures of MM_sec (secant form) and the plain aggregate + at rational tilt t, from scratch: own kernel, own census, own root and + log2 enclosures. Also: exact marginals, exact dichotomy, and an exact + per-row proof that margin-degenerate histories contribute 0.""" + u = rationalize(u_float) + supp, kern = exact_kernel(u, n, t) + m_at = len(supp) + Z = sum(sum(r) for r in kern) + # exact elementwise marginals + mu_row = [sum(r) for r in kern] + marg = [F0] * n + for idx, A in enumerate(supp): + for j in range(n): + if A >> j & 1: + marg[j] += Fraction(mu_row[idx], Z) + in_regime = all(m < Fraction(RECORD_NUM, RECORD_DEN) for m in marg) + + num_lo = num_hi = F0 # MM_sec numerator (unnormalized, /Z scale) + pl_lo = pl_hi = F0 # plain numerator: sum m log2(OR/t) + den = 0 # nondegenerate mass, integer scale + dich_ok = True + deg_zero_ok = True # degenerate rows provably contribute 0 + n_rows = 0 + zcache: dict[tuple[Fraction, Fraction], tuple[Fraction, Fraction]] = {} + for i in range(1, n): # i < n only + pmask, bit = (1 << (i - 1)) - 1, 1 << (i - 1) + has0, has1 = set(), set() + pref, resp = [], [] + for A in supp: + c = A & pmask + r = 1 if A & bit else 0 + pref.append(c) + resp.append(r) + (has1 if r else has0).add(c) + active = has0 & has1 + tables: dict[tuple[int, int], list[int]] = {} + for ia in range(m_at): + ca, ra, row = pref[ia], resp[ia], kern[ia] + for ib in range(m_at): + key = (ca, pref[ib]) + tab = tables.get(key) + if tab is None: + tab = tables[key] = [0, 0, 0, 0] + tab[(ra << 1) | resp[ib]] += row[ib] + for (a, b), (t00, t01, t10, t11) in tables.items(): + m = t00 + t01 + t10 + t11 + if m == 0: + continue + nondeg = t00 > 0 and t01 > 0 and t10 > 0 and t11 > 0 + if nondeg != (a in active and b in active): + dich_ok = False + x = Fraction(t00 + t01, m) + y = Fraction(t00 + t10, m) + z_rlz = Fraction(t00, m) + if not nondeg: + # exact convention-robustness check: margin degenerate and + # z~ equals the pinned boundary value of z_t for EVERY t + if x == 0 or y == 0: + if z_rlz != 0: + deg_zero_ok = False + elif x == 1: + if z_rlz != y: + deg_zero_ok = False + elif y == 1: + if z_rlz != x: + deg_zero_ok = False + else: + deg_zero_ok = False # nondeg margins but a zero cell + continue + key = (x, y) + if key not in zcache: + zlo, zhi = z_root_enc(x, y, t) + zcache[key] = h2_over(zlo, zhi) + htlo, hthi = zcache[key] + hrlo, hrhi = h2_enc(z_rlz) + num_lo += m * (hrlo - hthi) + num_hi += m * (hrhi - htlo) + q = Fraction(t00 * t11, t01 * t10) / t + llo, lhi = log2_enc(q) + pl_lo += m * llo + pl_hi += m * lhi + den += m + n_rows += 1 + return { + "mm_lo": num_lo / den, "mm_hi": num_hi / den, + "agg_lo": pl_lo / den, "agg_hi": pl_hi / den, + "max_marg": max(marg), "in_regime": in_regime, + "dich_ok": dich_ok, "deg_zero_ok": deg_zero_ok, + "n_rows": n_rows, "n_distinct_margins": len(zcache), + "den_over_Z": Fraction(den, Z), + } + + +# ====================================================================== +# float census (own implementation, from 007 section 1 + 018's prose) +# ====================================================================== + +def tilt(u: dict[int, float], lam: float) -> dict[tuple[int, int], float]: + z = 0.0 + pi = {} + for a, ua in u.items(): + for b, ub in u.items(): + v = ua * ub * 2.0 ** (lam * bin(a & b).count("1")) + pi[(a, b)] = v + z += v + return {k: v / z for k, v in pi.items()} + + +def z_pl(x: float, y: float, rho: float) -> float: + """Stable float Plackett root (own derivation: F(z)=0 with + F(z) = (1-rho) z^2 + (1-x-y+rho(x+y)) z - rho x y; the coupling root is + 2C' / (-B - sqrt(B^2-4AC)) written to avoid cancellation).""" + if rho == 1.0: + return x * y + A = 1.0 - rho + B = 1.0 - x - y + rho * (x + y) + C = -rho * x * y + disc = B * B - 4.0 * A * C + return -2.0 * C / (B + math.sqrt(disc)) + + +def h2f(z: float) -> float: + if z <= 0.0 or z >= 1.0: + return 0.0 + return -z * math.log2(z) - (1.0 - z) * math.log2(1.0 - z) + + +def sens(x: float, y: float, rho: float) -> float: + """d/d lambda of h2(z_rho(x,y)) at rho (own implicit differentiation: + z' = (x-z)(y-z) / [(1-x-y+2z) + rho(x+y-2z)]).""" + z = z_pl(x, y, rho) + if z <= 0.0 or z >= 1.0 or z >= min(x, y) or z <= max(0.0, x + y - 1.0): + return 0.0 + dz = (x - z) * (y - z) / ((1.0 - x - y + 2.0 * z) + + rho * (x + y - 2.0 * z)) + return math.log2((1.0 - z) / z) * dz * rho * math.log(2.0) + + +def census_variants(u: dict[int, float], n: int, lam: float): + """Own float census: plain aggregate, 018's three MM readings, and the + realized-OR variants 018 did not test. Returns None if no + nondegenerate history exists.""" + pi = tilt(u, lam) + supp = {a for (a, _b) in pi} + rho_t = 2.0 ** lam + acc = {k: 0.0 for k in + ("plain", "sec", "der", "absn", "der_rlz", "absn_rlz", + "absrho_rlz", "den", "denabs", "denabs_rlz", "denabsrho")} + dich = True + for i in range(1, n): + pmask, bit = (1 << (i - 1)) - 1, 1 << (i - 1) + has0, has1 = set(), set() + for A in supp: + (has1 if A & bit else has0).add(A & pmask) + active = has0 & has1 + tables: dict[tuple[int, int], list[float]] = {} + for (A, B), w in pi.items(): + key = (A & pmask, B & pmask) + tab = tables.setdefault(key, [0.0] * 4) + tab[(2 if A & bit else 0) + (1 if B & bit else 0)] += w + for (a, b), (p00, p01, p10, p11) in tables.items(): + nondeg = min(p00, p01, p10, p11) > 0.0 + if nondeg != (a in active and b in active): + dich = False + if not nondeg: + continue + m = p00 + p01 + p10 + p11 + x, y, z = (p00 + p01) / m, (p00 + p10) / m, p00 / m + l2or = (math.log2(p00) + math.log2(p11) + - math.log2(p01) - math.log2(p10)) + d = l2or - lam + rho = (p00 * p11) / (p01 * p10) + s_t = sens(x, y, rho_t) + s_r = sens(x, y, rho) + acc["plain"] += m * d + acc["sec"] += m * (h2f(z) - h2f(z_pl(x, y, rho_t))) + acc["der"] += m * s_t * d + acc["absn"] += m * abs(s_t) * d + acc["der_rlz"] += m * s_r * d + acc["absn_rlz"] += m * abs(s_r) * d + acc["absrho_rlz"] += m * (abs(s_r) / rho) * d + acc["den"] += m + acc["denabs"] += m * abs(s_t) + acc["denabs_rlz"] += m * abs(s_r) + acc["denabsrho"] += m * abs(s_r) / rho + if acc["den"] == 0.0: + return None + mu: dict[int, float] = {} + for (A, _B), w in pi.items(): + mu[A] = mu.get(A, 0.0) + w + marg = [sum(w for a, w in mu.items() if a >> j & 1) for j in range(n)] + return { + "plain": acc["plain"] / acc["den"], + "mm_sec": acc["sec"] / acc["den"], + "mm_der": acc["der"] / (acc["denabs"] or 1.0), + "mm_abs": acc["absn"] / (acc["denabs"] or 1.0), + "mm_der_rlz": acc["der_rlz"] / (acc["denabs_rlz"] or 1.0), + "mm_abs_rlz": acc["absn_rlz"] / (acc["denabs_rlz"] or 1.0), + "mm_absrho_rlz": acc["absrho_rlz"] / (acc["denabsrho"] or 1.0), + "max_marginal": max(marg), "dich_ok": dich, + "H": -sum(w * math.log2(w) for w in mu.values() if w > 0), + } + + +# ====================================================================== +# MU(n, r) ladder rebuilt from 016 section 1 prose (own code) +# ====================================================================== + +def mu_ladder(n: int, r: int, th: dict[str, float]) -> dict[int, float]: + """Coordinate 1 = block marker (bit 0); coords 2-4 = dilution singletons + (bits 1-3); response coords 5..4+r (bits 4..3+r); shared futures n-1, n + (bits n-2, n-1). Block a: {1,n-1}=A1, {1,n}=A2, light {1,c_j,n}; + block b: {}=B0, {n-1}=B1, {n}=B2, responders {c_j,n-1}. 2r+8 atoms.""" + assert 1 <= r <= n - 6 + hi, lo = 1 << (n - 1), 1 << (n - 2) + u = {1 | lo: th["A1"], 1 | hi: th["A2"], 0: th["B0"], + lo: th["B1"], hi: th["B2"]} + for j in range(r): + c = 1 << (4 + j) + u[1 | c | hi] = th["light"] + u[c | lo] = th["resp"] + for d in (1, 2, 3): + u[1 << d] = th["dil"] + return u + + +def marginals_u(u: dict[int, float], n: int, lam: float) -> list[float]: + supp = list(u) + z = 0.0 + mu = {} + for a in supp: + row = sum(u[a] * u[b] * 2.0 ** (lam * bin(a & b).count("1")) + for b in supp) + mu[a] = row + z += row + marg = [0.0] * n + for a, w in mu.items(): + for j in range(n): + if a >> j & 1: + marg[j] += w / z + return marg + + +def tune_dil(n: int, r: int, lam: float, th: dict[str, float] + ) -> dict[str, float]: + """016's dilution bisection re-implemented to spec: geometric bisection + of the shared dilution weight on [0.5, 2000], 18 steps, target max + marginal <= 0.375, dilution coords 2-4 vs the rest.""" + th = dict(th) + lo, hi = 0.5, 2000.0 + for _ in range(18): + w = math.sqrt(lo * hi) + th["dil"] = w + marg = marginals_u(mu_ladder(n, r, th), n, lam) + dil_m = max(marg[1:4]) + oth_m = max(m for j, m in enumerate(marg) if j not in (1, 2, 3)) + if oth_m > MARG_TARGET and dil_m < MARG_TARGET: + lo = w + elif dil_m > MARG_TARGET: + hi = w + else: + break + return th + + +# ====================================================================== +# S0 -- kit self-tests +# ====================================================================== + +def part_s0() -> dict: + out = {"ok": True} + + # log2 enclosure vs known values + cases = [(Fraction(1), 0.0), (Fraction(2), 1.0), (Fraction(1, 8), -3.0), + (Fraction(3), math.log2(3.0)), (Fraction(181, 16), + math.log2(181.0 / 16.0)), (Fraction(10) ** 20, + 20 * math.log2(10.0)), (Fraction(7, 5) ** 9, + 9 * math.log2(1.4))] + wmax = 0.0 + for q, ref in cases: + lo, hi = log2_enc(q) + wmax = max(wmax, float(hi - lo)) + out["ok"] &= bool(lo <= hi and float(lo) - 1e-12 <= ref + <= float(hi) + 1e-12) + rng = random.Random(19) + for _ in range(60): + q = Fraction(rng.randrange(1, 10 ** 12), rng.randrange(1, 10 ** 12)) + lo, hi = log2_enc(q) + ref = math.log2(q.numerator) - math.log2(q.denominator) + wmax = max(wmax, float(hi - lo)) + out["ok"] &= bool(float(lo) - 1e-9 <= ref <= float(hi) + 1e-9) + a, b = Fraction(355, 113), Fraction(1234567, 7654321) + la, ha = log2_enc(a) + lb, hb = log2_enc(b) + lab, hab = log2_enc(a * b) + out["ok"] &= bool(la + lb <= hab and lab <= ha + hb) + out["log2_max_width"] = wmax + log(f"[S0] log2 enclosure: {len(cases) + 60} cases + additivity, " + f"max width {wmax:.1e}, ok={out['ok']}") + + # Plackett root: closed forms + float cross-check + monotonicity in rho + ok_root = True + wr = 0.0 + for _ in range(60): + x = Fraction(rng.randrange(1, 99), 100) + y = Fraction(rng.randrange(1, 99), 100) + t = Fraction(rng.randrange(101, 4000), 100) + zlo, zhi = z_root_enc(x, y, t) + wr = max(wr, float(zhi - zlo)) + zf = z_pl(float(x), float(y), float(t)) + ok_root &= bool(float(zlo) - 1e-9 <= zf <= float(zhi) + 1e-9) + ok_root &= bool(x * y <= zlo <= zhi <= min(x, y)) + # the realized-table identity: build the exact table with margins + # x, y and both-zero z*, check its OR is inside [t*(1-eps), ...] + zm = (zlo + zhi) / 2 + orr = (zm * (1 - x - y + zm)) / ((x - zm) * (y - zm)) + ok_root &= bool(abs(float(orr / t) - 1.0) < 1e-9) + out["root_ok"] = ok_root + out["root_max_width"] = wr + out["ok"] &= ok_root + log(f"[S0] Plackett Newton root: 60 random (x,y,t), max width {wr:.1e}, " + f"round-trip OR/t = 1 to 1e-9, ok={ok_root}") + + # h2 enclosure + ok_h = True + for _ in range(40): + z = Fraction(rng.randrange(1, 999), 1000) + lo, hi = h2_enc(z) + ok_h &= bool(float(lo) - 1e-12 <= h2f(float(z)) <= float(hi) + 1e-12) + l, h = h2_enc(HALF) + ok_h &= bool(float(l) <= 1.0 <= float(h) + 1e-30) + out["h2_ok"] = ok_h + out["ok"] &= ok_h + log(f"[S0] h2 enclosure: 41 cases ok={ok_h}") + + # ladder spec check: MU(7,1) with witness theta == the 10 witness atoms + lad = mu_ladder(7, 1, THETA_WITNESS) + ok_lad = lad == WITNESS_U + out["ladder_matches_witness"] = ok_lad + out["ok"] &= ok_lad + log(f"[S0] MU(7,1) from spec == 007 witness atom-for-atom: {ok_lad}") + + save("gap1csk_s0.json", out) + if not out["ok"]: + raise SystemExit("S0 kit self-test failure -- abort") + return out + + +# ====================================================================== +# S1 -- K1: exact certification, from scratch +# ====================================================================== + +def part_s1() -> dict: + out = {} + t = Fraction(181, 16) + r = exact_mm_cert(WITNESS_U, 7, t) + lo, hi = r["mm_lo"], r["mm_hi"] + verdict = ("CERTIFIED NEGATIVE" if hi < 0 else + "CERTIFIED POSITIVE" if lo > 0 else "STRADDLES 0") + log(f"[S1] witness t=181/16 MM_sec in [{float(lo):+.15e}, " + f"{float(hi):+.15e}] width {float(hi - lo):.1e} {verdict}") + log(f"[S1] plain aggregate in [{float(r['agg_lo']):+.12f}, " + f"{float(r['agg_hi']):+.12f}] (013/018: +1.844669005)") + log(f"[S1] max marginal {float(r['max_marg']):.10f} " + f"(exact < 38271/100000: {r['in_regime']}); dichotomy exact " + f"{r['dich_ok']}; degenerate rows contribute 0 exactly " + f"{r['deg_zero_ok']}; rows {r['n_rows']} " + f"({r['n_distinct_margins']} distinct margins)") + # containment check vs 018's certificate + lo18 = Fraction(-22254658205655785177, 10 ** 21) + hi18 = Fraction(-22254658205655785176, 10 ** 21) + overlap = not (hi < lo18 or lo > hi18) + log(f"[S1] overlap with 018's enclosure " + f"[-0.022254658205655785177, ...176]: {overlap}") + out["witness"] = { + "mm_lo": f"{float(lo):.18e}", "mm_hi": f"{float(hi):.18e}", + "mm_lo_str": str(lo.limit_denominator(10 ** 30)), + "width": float(hi - lo), + "agg_lo": float(r["agg_lo"]), "agg_hi": float(r["agg_hi"]), + "max_marg": float(r["max_marg"]), "in_regime": r["in_regime"], + "dich_ok": r["dich_ok"], "deg_zero_ok": r["deg_zero_ok"], + "n_rows": r["n_rows"], "verdict": verdict, + "overlaps_018_cert": overlap, + } + + # 2-digit tidy witness (018 part R quotes float -0.0223 at mm 0.3130) + u2 = {a: float(f"{w:.2g}") for a, w in WITNESS_U.items()} + r2 = exact_mm_cert(u2, 7, t) + verdict2 = ("CERTIFIED NEGATIVE" if r2["mm_hi"] < 0 else + "CERTIFIED POSITIVE" if r2["mm_lo"] > 0 else "STRADDLES 0") + log(f"[S1] tidy 2-digit witness t=181/16: MM_sec in " + f"[{float(r2['mm_lo']):+.9e}, {float(r2['mm_hi']):+.9e}] {verdict2} " + f"(max marg {float(r2['max_marg']):.4f}, in-regime " + f"{r2['in_regime']})") + out["tidy"] = {"mm_lo": float(r2["mm_lo"]), "mm_hi": float(r2["mm_hi"]), + "max_marg": float(r2["max_marg"]), + "in_regime": r2["in_regime"], "verdict": verdict2, + "dich_ok": r2["dich_ok"], "deg_zero_ok": r2["deg_zero_ok"]} + + # exact product-measure zero: product potential u(a) = w^{|a|} at + # rational w -- every nondegenerate history must have OR = t exactly, + # so the MM_sec numerator is exactly 0 term by term. Verified via the + # census: enclosure must contain 0 with tiny width, and the plain + # aggregate must be exactly 0 (OR/t = 1 rational -> log enclosure + # degenerate at 0). + n_p = 5 + w = Fraction(3, 7) + up = {a: w ** bin(a).count("1") for a in range(1 << n_p)} + rp = exact_mm_cert(up, n_p, Fraction(4)) + zero_in = rp["mm_lo"] <= 0 <= rp["mm_hi"] + plain_zero = rp["agg_lo"] == 0 == rp["agg_hi"] + log(f"[S1] product anchor (w=3/7, n=5, t=4): MM_sec enclosure contains " + f"0: {zero_in} (width {float(rp['mm_hi'] - rp['mm_lo']):.1e}); " + f"plain aggregate exactly 0: {plain_zero}") + out["product_anchor"] = {"zero_in": zero_in, "plain_exact_zero": + plain_zero, + "width": float(rp["mm_hi"] - rp["mm_lo"])} + save("gap1csk_s1.json", out) + return out + + +# ====================================================================== +# S2 -- K2: the definitional fork, incl. variants 018 did not test +# ====================================================================== + +def part_s2() -> dict: + out = {"witness": {}, "ladder96": {}} + cv = census_variants(WITNESS_U, 7, 3.5) + log("[S2] witness (n=7, lam=3.5) variant zoo:") + for k in ("plain", "mm_sec", "mm_der", "mm_abs", "mm_der_rlz", + "mm_abs_rlz", "mm_absrho_rlz"): + log(f"[S2] {k:14s} {cv[k]:+.6f}") + out["witness"][k] = cv[k] + out["witness"]["max_marginal"] = cv["max_marginal"] + + # the realized-OR readings on the theta* ladder kill instance + th = tune_dil(96, 90, 2.0, THETA_STAR_L2) + cl = census_variants(mu_ladder(96, 90, th), 96, 2.0) + log("[S2] theta* ladder n=96 lam=2 variant zoo:") + for k in ("plain", "mm_sec", "mm_der", "mm_abs", "mm_der_rlz", + "mm_abs_rlz", "mm_absrho_rlz"): + log(f"[S2] {k:14s} {cl[k]:+.6e}") + out["ladder96"][k] = cl[k] + out["ladder96"]["max_marginal"] = cl["max_marginal"] + + # small-lambda sign crossing of MM_sec on the witness (018: ~0.7) + prof = [] + for lam in (0.6, 0.7, 0.75, 0.8, 0.9): + c = census_variants(WITNESS_U, 7, lam) + prof.append({"lambda": lam, "mm_sec": c["mm_sec"]}) + log("[S2] witness MM_sec near the crossing: " + + " ".join(f"{p['lambda']:.2f}:{p['mm_sec']:+.2e}" for p in prof)) + out["crossing_profile"] = prof + save("gap1csk_s2.json", out) + return out + + +# ====================================================================== +# S3 -- K3: float spot-checks with the spec-rebuilt ladder +# ====================================================================== + +def part_s3() -> dict: + out = {} + # witness headline floats + cv = census_variants(WITNESS_U, 7, 3.5) + ok_abs = abs(cv["mm_abs"] - 2.354328) < 5e-4 + ok_der = abs(cv["mm_der"] - (-0.895542)) < 5e-4 + ok_sec = abs(cv["mm_sec"] - (-0.022255)) < 5e-5 + log(f"[S3] witness floats: MM_abs {cv['mm_abs']:+.6f} (018 +2.354328 " + f"{ok_abs}), MM_der {cv['mm_der']:+.6f} (018 -0.895542 {ok_der}), " + f"MM_sec {cv['mm_sec']:+.6f} (018 -0.022255 {ok_sec})") + out["witness"] = {"mm_abs": cv["mm_abs"], "mm_der": cv["mm_der"], + "mm_sec": cv["mm_sec"], "ok": ok_abs and ok_der + and ok_sec} + + # theta* ladder n=96 lam=2, own tune_dil + th = tune_dil(96, 90, 2.0, THETA_STAR_L2) + c96 = census_variants(mu_ladder(96, 90, th), 96, 2.0) + ok96 = (abs(c96["plain"] - (-0.000760)) < 5e-5 + and abs(c96["mm_sec"] - (-3.819e-4)) < 5e-6 + and abs(c96["mm_abs"] - 0.969) < 5e-3) + log(f"[S3] theta* ladder n=96 lam=2 (own tune_dil, dil=" + f"{th['dil']:.6f}): plain {c96['plain']:+.6f} (018 -0.000760), " + f"MM_sec {c96['mm_sec']:+.6e} (018 -3.819e-4), MM_abs " + f"{c96['mm_abs']:+.6f} (018 +0.969), mm {c96['max_marginal']:.3f}, " + f"ok={ok96}") + out["theta96"] = {"dil": th["dil"], "plain": c96["plain"], + "mm_sec": c96["mm_sec"], "mm_abs": c96["mm_abs"], + "max_marginal": c96["max_marginal"], "ok": ok96} + + # raw ladder n=96 lam=2 -- the sign 018's prose gets wrong for n >= 96 + thr = tune_dil(96, 90, 2.0, THETA_WITNESS) + cr = census_variants(mu_ladder(96, 90, thr), 96, 2.0) + log(f"[S3] raw ladder n=96 lam=2 (own tune_dil, dil={thr['dil']:.4f}): " + f"plain {cr['plain']:+.6f}, MM_sec {cr['mm_sec']:+.6e}, MM_der " + f"{cr['mm_der']:+.6e} <-- MM_sec/MM_der POSITIVE at n=96 (018 " + f"checkpoint agrees; 018 PROSE says negative at every n)") + out["raw96"] = {"dil": thr["dil"], "plain": cr["plain"], + "mm_sec": cr["mm_sec"], "mm_der": cr["mm_der"], + "max_marginal": cr["max_marginal"]} + # raw ladder n=64 for the branch contrast + thr64 = tune_dil(64, 58, 2.0, THETA_WITNESS) + cr64 = census_variants(mu_ladder(64, 58, thr64), 64, 2.0) + log(f"[S3] raw ladder n=64 lam=2: MM_sec {cr64['mm_sec']:+.6e} " + f"(negative branch, 018 -3.400e-3)") + out["raw64"] = {"mm_sec": cr64["mm_sec"], "plain": cr64["plain"]} + + # part-D window value at n=96 + fit arithmetic from 018's checkpoint + lw = LAMWIN_C / (96 - 3) + thw = tune_dil(96, 90, lw, THETA_STAR_L2) + cw = census_variants(mu_ladder(96, 90, thw), 96, lw) + ok_w = abs(cw["plain"] - 3.747e-3) < 5e-5 + log(f"[S3] theta* window n=96 lam={lw:.6f}: A {cw['plain']:+.6e} " + f"(018 +3.747e-3, ok={ok_w}), mm {cw['max_marginal']:.3f}") + out["window96"] = {"lamwin": lw, "agg": cw["plain"], + "max_marginal": cw["max_marginal"], "ok": ok_w} + + # a1/a2 fit arithmetic re-done from 018's stored points (data reuse) + fit_rows = [] + try: + d = json.loads((DATA / "gap1c_partD.json").read_text()) + for row in d["rows"]: + if "points" not in row: + continue + pts = row["points"] + p1, p2 = pts[-1], pts[-2] + l1, l2 = p1["lambda"], p2["lambda"] + det = l1 * l2 * (l2 - l1) + a1 = (p1["agg"] * l2 * l2 - p2["agg"] * l1 * l1) / det + a2 = (p2["agg"] * l1 - p1["agg"] * l2) / det + ok = (abs(a1 - row["a1"]) < 1e-9 * max(1, abs(row["a1"])) + and abs(a2 - row["a2"]) < 1e-9 * max(1, abs(row["a2"]))) + fit_rows.append({"name": row["name"], "n": row["n"], + "a1": a1, "a2": a2, "a1n": a1 * row["n"], + "match": ok}) + th_star = [r for r in fit_rows if r["name"].startswith("theta*")] + log("[S3] a1/a2 fit re-solve (theta*): " + + " ".join(f"n={r['n']}:a1*n={r['a1n']:+.3f}" + f"{'' if r['match'] else '(MISMATCH)'}" + for r in th_star)) + all_match = all(r["match"] for r in fit_rows) + log(f"[S3] all 10 stored (a1, a2) rows re-derive: {all_match}; " + f"a1*n growth on theta* rows: " + f"{[round(r['a1n'], 2) for r in th_star]}") + out["fit_rows"] = fit_rows + out["fit_all_match"] = all_match + except FileNotFoundError: + log("[S3] gap1c_partD.json missing -- fit check skipped") + + # perturbation battery, DIFFERENT seeds than 018 (which used 2024) + res = {} + for seed in (7, 424242): + rng = random.Random(seed) + vals = [] + nfail = 0 + for _ in range(20): + u = {a: w * math.exp(rng.uniform(-0.03, 0.03)) + for a, w in WITNESS_U.items()} + c = census_variants(u, 7, 3.5) + vals.append(c["mm_sec"]) + if c["mm_sec"] >= 0 or c["max_marginal"] > 0.38271: + nfail += 1 + res[seed] = {"n_fail": nfail, "min": min(vals), "max": max(vals)} + log(f"[S3] perturbations seed={seed}: {nfail}/20 failures, MM_sec " + f"in [{min(vals):+.5f}, {max(vals):+.5f}]") + out["perturbations"] = res + save("gap1csk_s3.json", out) + return out + + +# ====================================================================== +# S4 -- K4: scope checks +# ====================================================================== + +def part_s4() -> dict: + out = {} + # best_free endpoint from 018 part C (data) + try: + d = json.loads((DATA / "gap1c_partC.json").read_text()) + bf = d["best_free"] + u = {int(k, 2): float(w) for k, w in bf["u"].items()} + c = census_variants(u, 8, bf["lambda"]) + ok = (abs(c["mm_abs"] - bf["mm_abs"]) < 1e-9 + and abs(c["max_marginal"] - bf["max_marginal"]) < 1e-9) + log(f"[S4] best_free endpoint (n=8, lam={bf['lambda']}): my MM_abs " + f"{c['mm_abs']:+.6e} (018 {bf['mm_abs']:+.6e}), MM_sec " + f"{c['mm_sec']:+.6e}, max marg {c['max_marginal']:.6f} " + f"(in-regime {c['max_marginal'] < 0.38271}), dich " + f"{c['dich_ok']}, H {c['H']:.3f} bits, match={ok}") + out["best_free"] = {"mm_abs": c["mm_abs"], "mm_sec": c["mm_sec"], + "max_marginal": c["max_marginal"], + "dich_ok": c["dich_ok"], "H": c["H"], + "match": ok, + "in_regime": c["max_marginal"] < 0.38271} + except FileNotFoundError: + log("[S4] gap1c_partC.json missing -- best_free check skipped") + + # the z = 1/2 sensitivity boundary vs the golden-ratio parenthetical. + # 018 claims z_1(x,x) = x^2 crosses 1/2 "exactly at the (3-sqrt5)/2 + # barrier". Arithmetic: x^2 = 1/2 at x = 2^-1/2 = 0.70711, i.e. + # element marginal 1 - 2^-1/2 = 0.29289, NOT 0.38197. At the barrier + # x = 1 - (3-sqrt5)/2 = (sqrt5-1)/2 and x^2 = (3-sqrt5)/2 = the + # marginal itself (phi^2 = 1 - phi), not 1/2. + s5 = math.sqrt(5.0) + barrier = (3.0 - s5) / 2.0 + x_bar = 1.0 - barrier + out["golden"] = { + "barrier": barrier, + "x_at_barrier": x_bar, + "x2_at_barrier": x_bar * x_bar, + "claim_x2_equals_half": abs(x_bar * x_bar - 0.5) < 1e-9, + "x2_equals_marginal": abs(x_bar * x_bar - barrier) < 1e-12, + "true_crossing_marginal_rho1": 1.0 - math.sqrt(0.5), + } + log(f"[S4] golden parenthetical: at marginal {barrier:.6f}, x = " + f"{x_bar:.6f}, x^2 = {x_bar * x_bar:.6f} = the marginal itself " + f"(phi identity), NOT 1/2; z_1(x,x) = 1/2 at marginal " + f"{1.0 - math.sqrt(0.5):.6f}") + # where DOES z_t(x,x) = 1/2 at t = 2^3.5? solve 3/4 - x = t (x-1/2)^2 + t = 2.0 ** 3.5 + lo, hi = 0.5, 0.75 + for _ in range(60): + mid = (lo + hi) / 2 + if z_pl(mid, mid, t) < 0.5: + lo = mid + else: + hi = mid + out["golden"]["crossing_marginal_at_lam3.5"] = 1.0 - lo + log(f"[S4] z_t(x,x) = 1/2 at lam=3.5 happens at element marginal " + f"{1.0 - lo:.6f} (lambda-dependent; equals 0.38197 for no stated " + f"reason)") + save("gap1csk_s4.json", out) + return out + + +def main() -> None: + part_s0() + part_s1() + part_s2() + part_s3() + part_s4() + log("done") + + +if __name__ == "__main__": + main() diff --git a/problems/union-closed/prior-art.json b/problems/union-closed/prior-art.json index 70caa01..82297a1 100644 --- a/problems/union-closed/prior-art.json +++ b/problems/union-closed/prior-art.json @@ -596,6 +596,95 @@ "252 atoms not 253" ], "note": "Decisive check per the 2026-07-31 verification standard: 014's certificates reused 013's exact engine and enclosures, so this review removes the shared-code risk with a third census written from 007 section 1's prose and a structurally different certified log2 enclosure (dyadic interval squaring; ln 2 from the exact series sum 1/(k 2^k)). Free anchors verified exactly: OR/t = 1 at i = n (005 Prop 1) on every instance, and including the i = n term keeps every kill negative. Union-closure of the support is correctly NOT a hypothesis (the control quantifies over all mu; 007's witness was not union-closed either) -- checked and recorded so the scope is explicit. Precedent followed from 013/007: the reviewed entry (014) is not modified." + }, + { + "id": "018", + "file": "attempts/018-surviving-controls-vs-ladder.md", + "date": "2026-08-06", + "mode": "informed", + "status": "REFUTED", + "mechanism": [ + "coupling", + "overlap-tilt", + "sinkhorn", + "plackett", + "exact-rational-arithmetic", + "certified-enclosure", + "chain-rule-assembly" + ], + "one_line": "Triage of the two post-016 Gap-1 candidates against the ladder adversary: the margin-modulated OR control is REFUTED in both signed readings already at n = 7 by 007's own witness (secant form certified -0.0222546582 at t = 181/16, in-regime -- the h-sensitivity weight is signed and kills the surplus histories, not the hidden deficit), while the unsigned |sigma|-weighted variant and the lambda-window-restricted control survive theta-re-optimized ladders, free-support climbs and joint (theta, lambda) attacks, with the window margin's first-order coefficient a1(n)*n GROWING in n.", + "range": "Witness kill certified exact at t = 181/16 (dyadic-bisection Plackett-root enclosures, directed log2 entropy enclosures), perturbation-stable 20/20 at 3%, 2-digit tidy, violating for every lambda in [1, 5]; MM variants measured on raw and theta* ladders n <= 160 (incl. 016's three certified kill instances), theta re-optimized against MM_abs at n = 48 with transfers to 160, 8 free-support climbs at n = 8; window probe at lam = 4.847/(n-3), /2, /4 on ladders n <= 320 with theta re-optimized at window lambda and a joint (theta, lam <= window) climb at n = 96", + "refuted_by": null, + "gaps": [ + "abs-sensitivity-or-control", + "lambda-restricted-or-control", + "mutual-information-tax", + "recipe-totality" + ], + "leak_terms": [ + "margin-modulated", + "MM_sec", + "MM_der", + "MM_abs", + "secant form", + "Plackett sensitivity", + "h-sensitivity", + "sensitivity sign flip", + "z crosses 1/2", + "0.0222546582", + "uc_gap1_candidates", + "gap1c", + "a1(n)*n", + "window margin coefficient", + "abs-sensitivity-or-control" + ], + "note": "Settles 016 lead 1 negatively for the signed readings without needing the ladder: the sensitivity weighting fails on the witness itself while the plain aggregate is certified positive (+1.844669) on the same tables. The margin-modulated gap tag of 007/013/016 dies as stated; what replaces it is the unsigned variant (which decouples from the chain rule -- a proof of it would not feed the 008/012 assembly directly) and the window-restricted variant (016 lead 4 / 017 C2), both surviving as EVIDENCE. Certification of the kill uses the lab's first certified enclosure of a NONLINEAR census functional (Plackett root + binary entropy per history)." + }, + { + "id": "019", + "file": "attempts/019-skeptic-review-of-018.md", + "date": "2026-08-06", + "mode": "informed", + "status": "VERIFIED_WITH_CORRECTIONS", + "mechanism": [ + "adversarial-verification", + "independent-reimplementation", + "exact-rational-arithmetic", + "certified-enclosure", + "coupling", + "overlap-tilt", + "sinkhorn", + "plackett" + ], + "one_line": "From-scratch re-certification confirms 018's headline: an independent exact kit (scaled-integer census from 007 section 1's prose, exact-rational Newton with sign-checked bracket for the Plackett root, own interval-squaring log2 enclosures) re-derives MM_sec in [-0.022254658205655785176146, -0.022254658205655785176145] on the witness at t = 181/16 -- negative, in-regime, dichotomy exact, strictly inside 018's enclosure -- with the secant identity, product-measure zero and degenerate-contribution-0 re-derived by hand and the tidy 2-digit witness newly certified (-0.02227); the realized-OR sensitivity variants 018 skipped were built and tested (signed dies on the witness at -1.458, unsigned survives), so no signed reading of 007 lead 2 survives; corrections are prose-level only.", + "leak_terms": [ + "gap1csk", + "uc_gap1c_skeptic", + "sign-checked Newton bracket", + "realized-OR variant", + "MM_der_rlz", + "MM_abs_rlz", + "0.022254658205655785176", + "raw ladder branch flip", + "golden parenthetical" + ], + "verifies": "018", + "corrections": [ + "018 part B's 'MM_sec is negative at every n in {16..160} (raw and theta* ladders), MM_der negative throughout' is contradicted by 018's own gap1c_partB.json: the raw ladder at n in {96, 128, 160} has MM_sec +6.7e-3/+6.1e-3/+5.2e-3 and MM_der ~ +0.95..0.99 (the tune_dil bisection switches dilution branch at n >= 96); correct scope is theta* at all n and raw at n <= 64 only", + "the golden-ratio parenthetical (twice: 'z_1(x,x) = x^2 crosses 1/2 exactly at the (3-sqrt5)/2 barrier') is false -- at the barrier x^2 equals the marginal itself (phi^2 = 1 - phi), not 1/2; the rho = 1 crossing is at marginal 1 - 2^{-1/2} ~ 0.2929 and the lambda = 3.5 crossing at ~ 0.3891 is lambda-dependent; the in-regime z = 1/2 straddling claim itself survives", + "section F ships a literal PENDING-F placeholder: only 1 of 4 --cert jobs (the witness MM cert) had landed at review time; the n = 96/160 MM certificates and t = 51/50 window certificates are absent from 018 and must not be cited from it", + "witness MM_sec 'positive only for lambda <~ 0.7' -- the crossing is between 0.8 and 0.9 (+4.95e-5 at 0.8, -3.2e-4 at 0.9)", + "runtime drift (cosmetic): record says ~35 min for parts A-D, log shows ~58 min" + ], + "range": "Witness MM_sec kill independently re-certified at t = 181/16 (enclosure width 5.9e-30, strictly inside 018's; exact marginals < 38271/100000; dichotomy and degenerate-row-zero checked as exact identities); tidy 2-digit witness newly certified negative; exact product-measure anchor (w = 3/7, n = 5, t = 4) contains 0 at width 9.3e-30 with plain aggregate exactly 0; float spot-checks on a spec-rebuilt ladder (own builder + own tune_dil): witness MM numbers, theta* n = 96 lam = 2 triple, window A at n = 96, all 10 part-D a1/a2 fits re-solved, perturbations 0/40 failures on fresh seeds; best_free endpoint re-evaluated (in-regime, nondegenerate); realized-OR variant zoo at two instances; searches themselves NOT re-run at scale (018's floors stay EVIDENCE); MM_der not separately certified (float only, as in 018)", + "refuted_by": null, + "gaps": [ + "abs-sensitivity-or-control", + "lambda-restricted-or-control", + "mutual-information-tax", + "recipe-totality" + ], + "note": "Decisive check per the lab standard: 018's certificate reused 013's atanh log2 kit and its own dyadic bisection; this review's kit shares neither (interval-squaring log2 re-implemented from 017's idea; exact Newton root with per-call symbolic bracket-sign identities F(xy) = (1-t)xy(1-x)(1-y) < 0 < min(x,y)(1-max(x,y))). Three structurally different kits now agree on the witness tables (013's aggregate, 018's MM, this one). The fork analysis is extended, not just checked: sensitivity-at-realized-OR readings were the natural loophole and they behave exactly as 018's signed/unsigned dichotomy predicts. Per precedent the reviewed record (018) is not modified." } ] }