diff --git a/.claude/blackboard.md b/.claude/blackboard.md index f4e502aa..60c9de36 100644 --- a/.claude/blackboard.md +++ b/.claude/blackboard.md @@ -3,7 +3,52 @@ > **Read this first.** The "Polyglot Notebook" architecture below is a > separate/older program, not the current epoch. -## 2026-07-06 (latest) — F64 GEMM completed: FMA tier + register residency + native-engine swap +## 2026-07-29 (latest) — blake3 dependency DROPPED: call sites swapped to the in-tree module + +Operator: "go ahead" on the rung-3a swap, with the measured cost table in +hand. This closes the only rung of `the-simd-ladder.md` that carried a cargo +cycle (`blake3 → ndarray::simd`). + +1. **15 call sites, 8 modules** (`seal`, `merkle_tree`, `plane`, `vsa`, + `spo_bundle`, `crystal_encoder`, `compression_curves`, `deepnsm`) now + reach `crate::hpc::blake3`. Implemented as one `use super::blake3;` per + module rather than rewriting each `blake3::` path — the in-tree API is + shape-compatible (`hash`, `Hasher::{new,new_keyed,new_derive_key,update, + finalize,finalize_xof}`, `Hash::as_bytes → &[u8;32]`), so the diff is 10 + added lines, not 15 edited ones. `spo_bundle`'s use is inside + `#[cfg(test)]`, so it takes `use crate::hpc::blake3;` there instead of + leaning on the subtree's blanket `unused_imports` allow. +2. **Dependency gone.** `dep:blake3` off the `std` feature; the ~55-line + Cargo.toml comment block replaced with a do-not-re-add note. `blake3` + **and** its transitive `constant_time_eq` are both absent from + `Cargo.lock`. nostd matrix untouched — every call site is under + `pub mod hpc`, itself `#[cfg(feature = "std")]`. +3. **What this actually bought:** the second SIMD surface. Even at + `default-features = false, features = ["pure"]` the crate shipped its own + `rust_{sse2,sse41,avx2}.rs` intrinsics beside `ndarray::simd` — the exact + thing the matryoshka pattern exists to prevent. The old Cargo.toml comment + named this and said "Tracked, not done here." + +**Cost, accepted with numbers in hand [MEASURED, not re-measurable]:** +1.25–1.39× across the input sizes this crate hashes (16 B 1.34–1.39×, 256 B +1.25–1.29×, 2 KB 1.30×); 4.7–4.9× at 64 KB, which no call site here touches. +The A/B bench needs both implementations present and is unbuildable from this +commit forward, by its own design note. + +**Loose ends.** +- `AdaWorldAPI/BLAKE3` (v1.8.5) exists and is an **unmodified upstream + mirror** — no ndarray wiring, all `ffi_*`/`rust_*` files intact. Master had + been pinning `blake3 = { version = "1" }` from **crates.io** while that fork + existed, i.e. the P0 fork rule was already broken. This change resolves it + by deletion; wiring the fork instead would have been the other resolution + and would NOT have closed the second-surface problem. +- Verified `src/hpc/blake3_test_vectors.json` is **byte-identical** (31,922 B) + to that fork's `test_vectors.json`, so the correctness tests run against + genuine upstream vectors. +- Rung 3b (`hash_many`) still absent; that is the whole 64 KB gap. Not worth + building for ndarray's own call sites — nothing here hashes bulk. + +## 2026-07-06 — F64 GEMM completed: FMA tier + register residency + native-engine swap Operator: "complete the F64 gemm… and pr". Three moves, all on the entry below's foundation: diff --git a/.claude/knowledge/blake3-in-tree-measured.md b/.claude/knowledge/blake3-in-tree-measured.md index 04b46217..ce8c798b 100644 --- a/.claude/knowledge/blake3-in-tree-measured.md +++ b/.claude/knowledge/blake3-in-tree-measured.md @@ -1,8 +1,22 @@ # In-tree BLAKE3 — correct, and what it costs -> **Status: MEASURED, 2026-07-29.** Correctness against the official vectors; -> throughput against the external crate. Both numbers below are reproducible -> with the instruments in this directory. +> **Status: MEASURED, 2026-07-29 — and SWAPPED, 2026-07-29.** Correctness +> against the official vectors; throughput against the external crate. +> +> **The open decision this document existed to inform has been taken:** the +> operator authorised the swap, all 15 call sites now reach +> `crate::hpc::blake3`, and the external dependency is gone. The cost below +> was accepted with the numbers in hand, not discovered afterward. +> +> Two consequences for anyone reading this later: +> +> - The **correctness** numbers remain reproducible. The vectors file +> (`src/hpc/blake3_test_vectors.json`) is byte-identical to upstream's +> `test_vectors.json` — verified against `AdaWorldAPI/BLAKE3` v1.8.5. +> - The **throughput** numbers are now historical and cannot be re-run. The +> A/B bench needs both implementations present; with the crate dropped there +> is nothing to compare against, exactly as its own header anticipated. To +> re-measure, you must temporarily re-add the dependency. ## READ BY: - Anyone about to drop the external `blake3` dependency @@ -21,10 +35,16 @@ cycle — the only rung of the ladder that has one (`the-simd-ladder.md`). Cutting it means ndarray owning BLAKE3 rather than consuming the crate. Scoping finding that made this small: **ndarray's usage is entirely -single-input.** 14 call sites across 8 files use only `hash`, -`Hasher::{new, new_keyed, update, finalize, finalize_xof().fill()}`, -`Hash::as_bytes`, and `Hash` as a signature type. **No `hash_many`.** So the -serial core suffices, and it needs no SIMD at all. +single-input.** 15 `blake3::` references across 8 files — 14 calls plus one +type position (`merkle_tree::truncate_hash`'s `&blake3::Hash` parameter) — +using only `hash`, `Hasher::{new, new_keyed, update, finalize, +finalize_xof().fill()}`, `Hash::as_bytes`, and `Hash` as a signature type. +**No `hash_many`.** So the serial core suffices, and it needs no SIMD at all. + +(An earlier revision said "14 call sites", counting calls but not the type +position, while the swap record above counts all 15 references. Raised by +CodeRabbit on #269; both numbers were describing different things, and the +distinction is now explicit rather than a discrepancy.) ## Correctness — proven @@ -135,9 +155,16 @@ hash small nodes, `deepnsm`/`compression_curves` small. So ndarray's real exposure is the **1.3–1.6× band**, not the 5× one — but 1.3× on a hot encoder path is a real cost, not a rounding error. -**Not swapped here.** The module lands and is tested; the call sites still -use the external crate. Flipping them is a decision with a measured price -tag, and it is the operator's. +**Not swapped in the commit this section was written for.** The module landed +and was tested; the call sites still used the external crate. Flipping them +was a decision with a measured price tag, and it was the operator's. + +**Taken 2026-07-29.** The operator authorised the flip with this table in +hand. All 15 call sites across the eight modules named above now reach +`crate::hpc::blake3` via a per-module `use super::blake3;`; `blake3` and its +transitive `constant_time_eq` are gone from `Cargo.lock`. The accepted price +is the 1.3× small-input band — the 4.8× 64 KB row was never on ndarray's +curve, since no call site here hashes bulk input. ## One deliberate deviation from upstream, and one restored diff --git a/Cargo.lock b/Cargo.lock index 378af986..fa97f8dc 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -84,18 +84,6 @@ dependencies = [ "password-hash", ] -[[package]] -name = "arrayref" -version = "0.3.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb" - -[[package]] -name = "arrayvec" -version = "0.7.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c02d123df017efcdfbd739ef81735b36c5ba83ec3c59c80a9d7ecc718f92e50" - [[package]] name = "autocfg" version = "1.5.0" @@ -150,20 +138,6 @@ dependencies = [ "digest 0.10.7", ] -[[package]] -name = "blake3" -version = "1.8.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4d2d5991425dfd0785aed03aedcf0b321d61975c9b5b3689c774a2610ae0b51e" -dependencies = [ - "arrayref", - "arrayvec", - "cc", - "cfg-if", - "constant_time_eq", - "cpufeatures 0.3.0", -] - [[package]] name = "blas-mock-tests" version = "0.1.0" @@ -396,12 +370,6 @@ version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" -[[package]] -name = "constant_time_eq" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b" - [[package]] name = "core-foundation" version = "0.10.1" @@ -1318,7 +1286,6 @@ name = "ndarray" version = "0.17.2" dependencies = [ "approx", - "blake3", "cblas-sys", "cranelift-codegen", "cranelift-frontend", diff --git a/Cargo.toml b/Cargo.toml index a93181ae..1ab8a142 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -154,63 +154,48 @@ libc = { version = "0.2.82", optional = true } matrixmultiply = { version = "0.3.2", default-features = false, features=["cgemm"] } # ===================================================================== -# blake3 — DEPENDENCY OF `std` FEATURE (NOT `hpc-extras`). READ BEFORE -# TOUCHING. +# blake3 — NO LONGER A DEPENDENCY. Do not re-add it. # ===================================================================== # -# blake3 is required by the cognitive substrate modules — plane, seal, -# merkle_tree, vsa, spo_bundle, crystal_encoder, compression_curves, -# deepnsm — which all live under `pub mod hpc;` (itself gated on -# `feature = "std"` in lib.rs). Those modules `use blake3;` directly -# and unconditionally; there is no #[cfg] dance inside them. +# BLAKE3 now lives in-tree at `src/hpc/blake3.rs`, as a **portable-only +# transcription of the BLAKE3 reference implementation -- no SIMD, no +# unsafe**. See that module's own header. It is the serial path every +# conformant implementation reduces to at `simd_degree() == 1`, which +# suffices here because ndarray's usage is entirely single-input. # -# Pinning blake3 to the `std` feature (rather than `hpc-extras`) means: -# * ANY consumer that enables `std` automatically gets blake3, with -# ZERO additional feature wiring. This is the default. Cargo's -# transitive feature resolution does the rest. -# * Consumers selecting `default-features = false, features = ["std"]` -# (e.g. burn-ndarray, which disables hpc-extras to shed p64/fractal) -# STILL get blake3 — no more "missing blake3" build errors that -# used to require chasing down hpc-extras. -# * Consumers selecting `default-features = false` (no std at all, -# e.g. the `thumbv6m-none-eabi` CI matrix entry) do NOT pull blake3 -# and therefore do not pull `constant_time_eq`, whose lack of -# `#![no_std]` declaration would otherwise fail the nostd link with -# `error[E0463]: can't find crate for std`. +# The external crate was removed because even with `default-features = +# false, features = ["pure"]` it carried its own SSE2/SSE4.1/AVX2 +# intrinsics -- a second SIMD surface beside `ndarray::simd`, which is +# exactly what the matryoshka pattern exists to prevent. The old comment +# block here said "Tracked, not done here"; this is it being done. # -# If you find yourself wanting to make blake3 unconditional (drop the -# `optional = true`), check first: does it still pass -# `cargo rustc -p ndarray --target=thumbv6m-none-eabi -# --no-default-features --features portable-atomic-critical-section`? -# If not, leave the `optional = true` + `std`-feature pinning in place. +# It also breaks a cycle. `ndarray -> blake3` meant blake3 could never be +# built ON `ndarray::simd` without cargo seeing `ndarray -> blake3 -> +# ndarray`. Dropping the dep is what makes an ndarray-backed BLAKE3 +# possible -- NOT here (this module is portable), but for a future +# SIMD-backed implementation and for the AdaWorldAPI/BLAKE3 fork. # -# ===================================================================== -# `default-features = false` + `pure`: NO C/ASM FFI. Operator directive -# 2026-07-28 — "I don't want any ffi with c". -# -# blake3's default build runs `cc::Build` over `c/blake3_{sse2,sse41,avx2, -# avx512}_x86-64_unix.S` and links them (measured before this change: 33 `.o` -# files plus `libblake3_avx512_assembly.a` in target/, with `blake3_*_ffi` -# cfgs set). The `pure` feature routes build.rs to -# `build_sse2_sse41_avx2_rust_intrinsics()` — its own comment: "No C code to -# compile here" — which only sets `blake3_{sse2,sse41,avx2}_rust` cfgs and -# lets the normal cargo build compile the Rust intrinsics modules. +# Separately: blake3 carried `cc` as an unconditional build-dependency, and +# the `pure` feature existed only to stop that `cc` from compiling the +# bundled C/ASM (operator directive 2026-07-28, "I don't want any ffi with +# c"). In-tree there is no C to opt out of, and `cc` is no longer pulled on +# blake3's account. Dropping blake3 also removed `constant_time_eq`, +# `arrayref` and `arrayvec` from the lockfile. # -# What `pure` costs (build.rs:344-371): the hand-tuned x86-64 assembly is -# replaced by Rust intrinsics, the AVX-512 path is dropped entirely, and the -# aarch64 NEON C intrinsics are dropped. What it removes: every byte of C. +# Cost, measured: 1.25-1.39x across the input sizes this crate actually +# hashes (16 B word 1.34-1.39x, 256 B 1.25-1.29x, 2 KB 1.30x), rising to +# 4.7-4.9x at 64 KB where upstream's `hash_many` has no in-tree counterpart. +# Every call site here sits in the small-input band, not the 64 KB one. Full +# table, methodology and the attribution of the gap: +# `.claude/knowledge/blake3-in-tree-measured.md`. # -# `default-features = false` also drops blake3's own `std` (which only gates -# `constant_time_eq/std`); this crate's `std` feature is what pulls blake3 in -# at all, so nothing here needs blake3's. +# The A/B bench under `.claude/knowledge/blake3-ab-bench/` compared the two +# implementations and is, by its own design note, unbuildable from this commit +# onward — there is no longer a second implementation to compare against. # -# NOTE: `pure` still leaves blake3's Rust SSE2/SSE4.1/AVX2 intrinsics — a -# second SIMD surface beside `ndarray::simd`, which the matryoshka pattern -# exists to prevent. Closing that means implementing BLAKE3's compression on -# `ndarray::simd::U32x16` (it is a ChaCha-derived u32 ARX kernel, and that -# lane is proven at the AVX2 instruction floor — see -# `.claude/knowledge/td-t22-asm-investigation.md`). Tracked, not done here. -blake3 = { version = "1", optional = true, default-features = false, features = ["pure"] } +# Call sites live under `pub mod hpc;` (gated on `feature = "std"` in lib.rs), +# so the nostd matrix is untouched: `blake3` and its `constant_time_eq` +# dependency are both gone from Cargo.lock entirely. # p64 + fractal — specialized convergence / manifold math. Gated behind # `hpc-extras` since they pull in a dep tree burn-ndarray doesn't need. @@ -306,7 +291,7 @@ blas = ["dep:cblas-sys", "dep:libc"] serde = ["dep:serde"] -std = ["num-traits/std", "matrixmultiply/std", "dep:blake3"] +std = ["num-traits/std", "matrixmultiply/std"] rayon = ["dep:rayon", "std"] # Portable-SIMD backend (NIGHTLY ONLY). Routes `crate::simd::*` types @@ -345,13 +330,9 @@ nightly-simd = ["std"] runtime-dispatch = ["std"] # HPC extras: p64 palette/NARS bridge + fractal manifold. -# (blake3 was previously listed here; it is now part of `std` directly -# because the cognitive substrate modules under hpc/ that import blake3 -# are themselves `std`-gated. See the blake3 comment in [dependencies].) # These pull in a non-trivial dependency tree; downstream crates such as # burn-ndarray that only need the core array layer can disable this with -# `default-features = false` (and re-enable `std` explicitly if needed — -# blake3 will come along with `std`). +# `default-features = false` (and re-enable `std` explicitly if needed). hpc-extras = ["std", "dep:p64", "dep:fractal", "fractal/std"] matrixmultiply-threading = ["matrixmultiply/threading"] diff --git a/README.md b/README.md index 6c52d621..89d70c3d 100644 --- a/README.md +++ b/README.md @@ -210,20 +210,26 @@ cargo test ### Transitive dependencies of the `std` feature -Enabling the `std` feature (the default) pulls in **`blake3`** as a hard -transitive dependency. The cognitive substrate modules under `hpc/` — -`plane`, `seal`, `merkle_tree`, `vsa`, `spo_bundle`, `crystal_encoder`, -`compression_curves`, `deepnsm` — import `blake3` directly for integrity -hashing and XOF expansion, and there is no separate feature to enable it. -This was previously gated behind `hpc-extras`, which caused recurring -"missing blake3" build errors for consumers (e.g. `burn-ndarray`) that -selected `default-features = false, features = ["std"]` to shed the -`p64` / `fractal` dependency tree. Pinning blake3 to `std` removes that -footgun: any `std`-enabled build automatically gets blake3. +**None for hashing.** BLAKE3 is in-tree. + +The cognitive substrate modules under `hpc/` — `plane`, `seal`, +`merkle_tree`, `vsa`, `spo_bundle`, `crystal_encoder`, `compression_curves`, +`deepnsm` — use `hpc::blake3` for integrity hashing and XOF expansion. That +is a portable pure-Rust transcription of the BLAKE3 reference +implementation, shipped in this crate: no SIMD, no `unsafe`, no C, and no +build script. + +Earlier revisions pulled the external **`blake3`** crate here, first gated +behind `hpc-extras` (which caused recurring "missing blake3" build errors +for consumers such as `burn-ndarray` selecting +`default-features = false, features = ["std"]`), then pinned to `std`. +**Both are gone.** `blake3` and its transitive `constant_time_eq`, +`arrayref` and `arrayvec` no longer appear in the dependency graph at any +feature combination, so the footgun cannot recur. Consumers building `default-features = false` (no `std`, e.g. the -`thumbv6m-none-eabi` nostd target) skip both the `hpc` module and the -blake3 dep, so the nostd link is unaffected. +`thumbv6m-none-eabi` nostd target) skip the `hpc` module and the BLAKE3 +code with it, so the nostd link is unaffected. ## Ecosystem diff --git a/crates/burn/Cargo.lock b/crates/burn/Cargo.lock index 968eaa00..e8488701 100644 --- a/crates/burn/Cargo.lock +++ b/crates/burn/Cargo.lock @@ -17,6 +17,19 @@ version = "2.0.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa" +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "getrandom 0.3.4", + "once_cell", + "version_check", + "zerocopy", +] + [[package]] name = "aho-corasick" version = "1.1.4" @@ -47,12 +60,6 @@ version = "1.0.102" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c" -[[package]] -name = "arrayref" -version = "0.3.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb" - [[package]] name = "arrayvec" version = "0.7.6" @@ -119,16 +126,6 @@ version = "1.8.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06" -[[package]] -name = "bincode" -version = "2.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "36eaf5d7b090263e8150820482d5d93cd964a81e4019913c972f4edcc6edb740" -dependencies = [ - "serde", - "unty", -] - [[package]] name = "bit-set" version = "0.9.1" @@ -153,20 +150,6 @@ dependencies = [ "serde_core", ] -[[package]] -name = "blake3" -version = "1.8.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2468ef7d57b3fb7e16b576e8377cdbde2320c60e1491e961d11da40fc4f02a2d" -dependencies = [ - "arrayref", - "arrayvec", - "cc", - "cfg-if", - "constant_time_eq", - "cpufeatures 0.2.17", -] - [[package]] name = "blas-src" version = "0.10.0" @@ -220,8 +203,8 @@ dependencies = [ [[package]] name = "burn-backend" -version = "0.21.0-pre.2" -source = "git+https://github.com/tracel-ai/burn.git?rev=ed72d2b#ed72d2b125a364aff18aed2a53396c128e01cb42" +version = "0.21.0-pre.4" +source = "git+https://github.com/AdaWorldAPI/burn.git?rev=9b2b671#9b2b67127b0fbb5387021faf540b7b12b9c4e943" dependencies = [ "burn-std", "bytemuck", @@ -240,8 +223,8 @@ dependencies = [ [[package]] name = "burn-ir" -version = "0.21.0-pre.2" -source = "git+https://github.com/tracel-ai/burn.git?rev=ed72d2b#ed72d2b125a364aff18aed2a53396c128e01cb42" +version = "0.21.0-pre.4" +source = "git+https://github.com/AdaWorldAPI/burn.git?rev=9b2b671#9b2b67127b0fbb5387021faf540b7b12b9c4e943" dependencies = [ "burn-backend", "hashbrown 0.16.1", @@ -250,8 +233,8 @@ dependencies = [ [[package]] name = "burn-std" -version = "0.21.0-pre.2" -source = "git+https://github.com/tracel-ai/burn.git?rev=ed72d2b#ed72d2b125a364aff18aed2a53396c128e01cb42" +version = "0.21.0-pre.4" +source = "git+https://github.com/AdaWorldAPI/burn.git?rev=9b2b671#9b2b67127b0fbb5387021faf540b7b12b9c4e943" dependencies = [ "bytemuck", "bytes", @@ -261,6 +244,7 @@ dependencies = [ "num-traits", "serde", "smallvec", + "spin", ] [[package]] @@ -330,10 +314,37 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6f8d983286843e49675a4b7a2d174efe136dc93a18d69130dd18198a6c167601" dependencies = [ "cfg-if", - "cpufeatures 0.3.0", + "cpufeatures", "rand_core", ] +[[package]] +name = "ciborium" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "42e69ffd6f0917f5c029256a24d0161db17cea3997d185db0d35926308770f0e" +dependencies = [ + "ciborium-io", + "ciborium-ll", + "serde", +] + +[[package]] +name = "ciborium-io" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05afea1e0a06c9be33d539b876f1ce3692f4afea2cb41f740e7743225ed1c757" + +[[package]] +name = "ciborium-ll" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "57663b653d948a338bfb3eeba9bb2fd5fcfaecb9e199e87e1eda4d9e8b240fd9" +dependencies = [ + "ciborium-io", + "half", +] + [[package]] name = "cmake" version = "0.1.58" @@ -383,12 +394,6 @@ dependencies = [ "tiny-keccak", ] -[[package]] -name = "constant_time_eq" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b" - [[package]] name = "convert_case" version = "0.10.0" @@ -414,15 +419,6 @@ version = "0.8.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" -[[package]] -name = "cpufeatures" -version = "0.2.17" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280" -dependencies = [ - "libc", -] - [[package]] name = "cpufeatures" version = "0.3.0" @@ -489,8 +485,8 @@ checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5" [[package]] name = "cubecl" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "cubecl-core", "cubecl-cuda", @@ -502,15 +498,15 @@ dependencies = [ [[package]] name = "cubecl-common" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "backtrace", - "bincode", "bytemuck", "bytes", "cfg-if", "cfg_aliases", + "ciborium", "derive-new", "derive_more", "dirs", @@ -533,6 +529,7 @@ dependencies = [ "serde_bytes", "serde_json", "spin", + "toml", "tynm", "wasm-bindgen-futures", "web-time", @@ -541,8 +538,8 @@ dependencies = [ [[package]] name = "cubecl-core" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "bitflags", "bytemuck", @@ -567,8 +564,8 @@ dependencies = [ [[package]] name = "cubecl-cpp" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "bytemuck", "cubecl-common", @@ -583,8 +580,8 @@ dependencies = [ [[package]] name = "cubecl-cuda" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "bytemuck", "cubecl-common", @@ -600,8 +597,8 @@ dependencies = [ [[package]] name = "cubecl-ir" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "cubecl-common", "cubecl-macros-internal", @@ -621,8 +618,8 @@ dependencies = [ [[package]] name = "cubecl-macros" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "cubecl-common", "darling 0.23.0", @@ -637,8 +634,8 @@ dependencies = [ [[package]] name = "cubecl-macros-internal" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "darling 0.23.0", "proc-macro2", @@ -648,8 +645,8 @@ dependencies = [ [[package]] name = "cubecl-opt" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "cubecl-common", "cubecl-core", @@ -665,9 +662,10 @@ dependencies = [ [[package]] name = "cubecl-runtime" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ + "ahash", "async-channel", "bytemuck", "cfg-if", @@ -687,15 +685,14 @@ dependencies = [ "spin", "thiserror", "toml", - "variadics_please", "wasm-bindgen-futures", "web-time", ] [[package]] name = "cubecl-wgpu" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "async-channel", "bytemuck", @@ -711,13 +708,14 @@ dependencies = [ "hashbrown 0.16.1", "log", "sanitize-filename", + "wasm-bindgen-futures", "wgpu", ] [[package]] name = "cubecl-zspace" -version = "0.10.0-pre.2" -source = "git+https://github.com/tracel-ai/cubecl?rev=5b831a3cfac3eca0065fe0dbf57cddf5946d1586#5b831a3cfac3eca0065fe0dbf57cddf5946d1586" +version = "0.10.0-pre.4" +source = "git+https://github.com/tracel-ai/cubecl?rev=f60142ccc35dcbede6db2c28a1315ae4cccabdd1#f60142ccc35dcbede6db2c28a1315ae4cccabdd1" dependencies = [ "derive-new", "serde", @@ -1197,6 +1195,18 @@ dependencies = [ "wasi", ] +[[package]] +name = "getrandom" +version = "0.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd" +dependencies = [ + "cfg-if", + "libc", + "r-efi 5.3.0", + "wasip2", +] + [[package]] name = "getrandom" version = "0.4.2" @@ -1205,7 +1215,7 @@ checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555" dependencies = [ "cfg-if", "libc", - "r-efi", + "r-efi 6.0.0", "rand_core", "wasip2", "wasip3", @@ -1663,13 +1673,13 @@ checksum = "8d5439c4ad607c3c23abf66de8c8bf57ba8adcd1f129e699851a6e43935d339d" name = "ndarray" version = "0.17.2" dependencies = [ - "blake3", "cblas-sys", "libc", "matrixmultiply", "num-complex", "num-integer", "num-traits", + "paste", "portable-atomic", "portable-atomic-util", "rawpointer", @@ -2093,6 +2103,12 @@ dependencies = [ "proc-macro2", ] +[[package]] +name = "r-efi" +version = "5.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f" + [[package]] name = "r-efi" version = "6.0.0" @@ -2101,9 +2117,9 @@ checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" [[package]] name = "rand" -version = "0.10.0" +version = "0.10.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bc266eb313df6c5c09c1c7b1fbe2510961e5bcd3add930c1e31f7ed9da0feff8" +checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80" dependencies = [ "chacha20", "getrandom 0.4.2", @@ -2652,12 +2668,6 @@ version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" -[[package]] -name = "unty" -version = "0.0.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6d49784317cd0d1ee7ec5c716dd598ec5b4483ea832a2dced265471cc0f690ae" - [[package]] name = "ureq" version = "3.3.0" diff --git a/crates/wasm-simd-parity/Cargo.lock b/crates/wasm-simd-parity/Cargo.lock index f734f708..d694d469 100644 --- a/crates/wasm-simd-parity/Cargo.lock +++ b/crates/wasm-simd-parity/Cargo.lock @@ -2,75 +2,12 @@ # It is not intended for manual editing. version = 4 -[[package]] -name = "arrayref" -version = "0.3.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "76a2e8124351fda1ef8aaaa3bbd7ebbcb486bbcd4225aca0aa0d84bb2db8fecb" - -[[package]] -name = "arrayvec" -version = "0.7.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56" - [[package]] name = "autocfg" version = "1.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" -[[package]] -name = "blake3" -version = "1.8.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0aa83c34e62843d924f905e0f5c866eb1dd6545fc4d719e803d9ba6030371fce" -dependencies = [ - "arrayref", - "arrayvec", - "cc", - "cfg-if", - "constant_time_eq", - "cpufeatures", -] - -[[package]] -name = "cc" -version = "1.2.67" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e17dd265a7d0f31ef544e1b20e03add05d3b45b491b633b10d67145d2acc1a38" -dependencies = [ - "find-msvc-tools", - "shlex", -] - -[[package]] -name = "cfg-if" -version = "1.0.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" - -[[package]] -name = "constant_time_eq" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b" - -[[package]] -name = "cpufeatures" -version = "0.3.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8b2a41393f66f16b0823bb79094d54ac5fbd34ab292ddafb9a0456ac9f87d201" -dependencies = [ - "libc", -] - -[[package]] -name = "find-msvc-tools" -version = "0.1.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582" - [[package]] name = "fractal" version = "0.1.0" @@ -78,12 +15,6 @@ dependencies = [ "libm", ] -[[package]] -name = "libc" -version = "0.2.186" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" - [[package]] name = "libm" version = "0.2.16" @@ -104,7 +35,6 @@ dependencies = [ name = "ndarray" version = "0.17.2" dependencies = [ - "blake3", "fractal", "matrixmultiply", "num-complex", @@ -178,12 +108,6 @@ version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3" -[[package]] -name = "shlex" -version = "2.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" - [[package]] name = "wasm-simd-parity" version = "0.0.0" diff --git a/src/hpc/compression_curves.rs b/src/hpc/compression_curves.rs index 0c7bcb6c..22f0ec66 100644 --- a/src/hpc/compression_curves.rs +++ b/src/hpc/compression_curves.rs @@ -14,6 +14,8 @@ //! - **Product Quantization** (sub-vector k-means, simplified): PQ-lite //! - **Random Projection + Binarize**: JL-lemma based +use super::blake3; + // ============================================================================ // PRNG (self-contained, deterministic) // ============================================================================ diff --git a/src/hpc/crystal_encoder.rs b/src/hpc/crystal_encoder.rs index 26188f9a..f878bb55 100644 --- a/src/hpc/crystal_encoder.rs +++ b/src/hpc/crystal_encoder.rs @@ -8,6 +8,7 @@ //! 3. **Pure Crystal Encoding** — hash-based word encoding via a 65-entry //! NSM semantic-primes codebook, bundled into SPO nodes. +use super::blake3; use super::fingerprint::Fingerprint; use super::node::Node; use super::plane::Plane; diff --git a/src/hpc/deepnsm.rs b/src/hpc/deepnsm.rs index 86daf045..be7abdfe 100644 --- a/src/hpc/deepnsm.rs +++ b/src/hpc/deepnsm.rs @@ -7,6 +7,7 @@ //! Transcoded from Python DeepNSM (). //! Rust 1.94.0 · SIMD via `crate::simd::F32x16` · Arrow 57 / DataFusion 51 ready. +use super::blake3; use std::collections::{HashMap, HashSet}; use std::sync::LazyLock; diff --git a/src/hpc/merkle_tree.rs b/src/hpc/merkle_tree.rs index a1f77e44..d9f848f9 100644 --- a/src/hpc/merkle_tree.rs +++ b/src/hpc/merkle_tree.rs @@ -5,6 +5,7 @@ //! on 16Kbit Containers (CLAM, CHAODA, CHESS, CAKES, panCAKES, BNN, //! cascade) should also work on the 8Kbit tree at higher speed. +use super::blake3; use super::seal::MerkleRoot; /// Number of branches in the Merkle tree. diff --git a/src/hpc/plane.rs b/src/hpc/plane.rs index a94769ea..36a81d0c 100644 --- a/src/hpc/plane.rs +++ b/src/hpc/plane.rs @@ -9,6 +9,7 @@ //! //! SIMD wired through: all bulk operations delegate to ndarray's bitwise dispatch. +use super::blake3; use super::fingerprint::Fingerprint; use std::cell::RefCell; diff --git a/src/hpc/seal.rs b/src/hpc/seal.rs index 350ed222..b0d47cdb 100644 --- a/src/hpc/seal.rs +++ b/src/hpc/seal.rs @@ -3,6 +3,7 @@ //! [`MerkleRoot`] is a 48-bit truncated blake3 hash of data masked by alpha. //! [`Seal`] is the verification result: `Wisdom` (matches) or `Staunen` (differs). +use super::blake3; use super::plane::Plane; /// Integrity seal. Byte equality comparison of blake3 hashes. diff --git a/src/hpc/spo_bundle.rs b/src/hpc/spo_bundle.rs index 2b2f6665..28c649f1 100644 --- a/src/hpc/spo_bundle.rs +++ b/src/hpc/spo_bundle.rs @@ -260,6 +260,7 @@ fn to_ranks(v: &[f64]) -> Vec { #[cfg(test)] mod tests { use super::*; + use crate::hpc::blake3; use std::collections::HashSet; // ======================================================================== diff --git a/src/hpc/vsa.rs b/src/hpc/vsa.rs index 0515ef1e..0aed0e83 100644 --- a/src/hpc/vsa.rs +++ b/src/hpc/vsa.rs @@ -12,6 +12,8 @@ //! tier (FP16x32 / FP32x16 / F64x8). Matches the Binary16K / Vsa16k carrier //! shared with lance-graph-contract (`crystal::fingerprint`). +use super::blake3; + /// VSA dimensionality: 16,384 bits (Binary16K). pub const VSA_DIMS: usize = 16_384; diff --git a/src/lib.rs b/src/lib.rs index 2c787693..0b510eb5 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -485,16 +485,21 @@ pub mod backend; /// extra deps. Cognitive/research modules (p64_bridge, crystal_encoder, /// deepnsm, etc.) are gated behind `hpc-extras` inside `hpc/mod.rs`. /// -/// ## blake3 transitive dep -/// -/// Enabling `std` (the default) automatically pulls `blake3`, which the -/// cognitive substrate modules (`plane`, `seal`, `merkle_tree`, `vsa`, -/// `spo_bundle`, `crystal_encoder`, `compression_curves`, `deepnsm`) -/// import directly and unconditionally. There is no separate feature -/// to enable; `std` is enough. Consumers building `default-features = false` -/// without `std` (e.g. the `thumbv6m-none-eabi` nostd target) skip both -/// the `hpc` module and the blake3 dep. See the `blake3` comment block -/// in `Cargo.toml` for the rationale. +/// ## BLAKE3 is in-tree — there is no `blake3` dependency +/// +/// The cognitive substrate modules (`plane`, `seal`, `merkle_tree`, `vsa`, +/// `spo_bundle`, `crystal_encoder`, `compression_curves`, `deepnsm`) hash +/// with `hpc::blake3`, a portable pure-Rust transcription of the BLAKE3 +/// reference implementation that ships inside this crate. +/// +/// **Enabling `std` no longer pulls the external `blake3` crate.** It, and +/// its transitive `constant_time_eq`, `arrayref` and `arrayvec`, are gone +/// from the dependency graph entirely. +/// +/// Consumers building `default-features = false` without `std` (e.g. the +/// `thumbv6m-none-eabi` nostd target) skip the `hpc` module, and with it +/// the BLAKE3 code. See the `blake3` comment block in `Cargo.toml` for why +/// the dependency was dropped and what it cost. #[cfg(feature = "std")] #[allow(clippy::all, unused_imports, unused_variables, unused_mut, dead_code)] pub mod hpc;