Point safetensors messaging at CRAN 0.3.0 instead of a GitHub build

CLAUDE.md

@@ -299,7 +299,8 @@ Native torch modules will load weights directly from HuggingFace safetensors for
 model <- load_from_hf("stabilityai/stable-diffusion-xl-base-1.0")
 ```
 
-See cornyverse CLAUDE.md for safetensors package setup (use cornball-ai fork until PR merged).
+See cornyverse CLAUDE.md for safetensors package setup. CRAN safetensors
+0.3.0 contains the required fixes; keep downstream gates capability-probed.
 
 ## Roadmap
 

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: diffuseR
 Title: Functional Interface to Diffusion Models in R
-Version: 0.2.2.6
+Version: 0.2.2.7
 Authors@R: c(
     person("Troy", "Hernandez", email = "troy@cornball.ai", role = c("aut", "cre"),
            comment = c(ORCID = "0009-0005-4248-604X")),

NEWS.md

@@ -1,3 +1,66 @@
+# diffuseR 0.2.2.7
+
+* `recommend()` diagnosed the wrong safetensors capability for bf16. The
+  tier gate consults `.st_can_read()`, but the note it produced cited
+  mlverse/safetensors#11, which is the bfloat16 *write* fix. bfloat16
+  read worked on CRAN 0.2.1, so a reader that lacks it is not waiting on
+  #11, and the message sent users to an unrelated issue. `.st_update_note()`
+  now takes `mode`, and both call sites pass the capability they actually
+  gated on. float8 is unchanged: 0.2.1 had neither read nor write, so #13
+  is correct from both sides.
+
+  The suite had been holding this in place. Its read-mode assertion
+  matched `"safetensors#11"`, so the wrong reference was pinned by the
+  test rather than caught by it. The remedy-only guards added earlier in
+  this release could not catch it either: the message named the right
+  package to install while blaming the wrong missing feature.
+
+* safetensors 0.3.0 reached CRAN on 2026-08-21 with all four fixes
+  diffuseR had been routing users around: float8 support
+  (mlverse/safetensors#13), bfloat16 write (#11), the >2 GB offset
+  overflow (#14), and empty tensor names (#10). Every message that told
+  users to install a development build from GitHub now tells them to run
+  `install.packages("safetensors")`. That covers `recommend()`'s tier
+  note, the graceful fp8/bf16 fallback, both `flux_quantize()` errors, and
+  the >2 GB read breadcrumb.
+
+  The same sweep reached the documentation, which is where most of the
+  stale advice actually was: the `shard_bytes` help for `flux_quantize()`,
+  `ltx23_quantize_nf4()`, `ltx23_quantize_fp8()` and
+  `gemma3_quantize_nf4()` described the 1.9e9 default as what "stock CRAN
+  safetensors" can read and pointed at a fork for anything larger.
+  `README.md`, `vignette("performance-levers")`, `reshard_safetensors()`
+  and the `unet_safetensors`, `download_prebuilt` and
+  `convert_sd21_pt_to_diffusers` help pages carried variants of the same.
+
+  All of it now names the capability rather than a version: "needs the
+  overflow fix (mlverse/safetensors#14, which reached CRAN in 0.3.0)"
+  rather than "requires safetensors 0.3.0 or newer". That distinction is
+  the package's whole stance here, documented below: a version number
+  never separated a capable build from an incapable one, so prose making
+  0.3.0 the requirement contradicts the probes it sits beside. The >2 GB
+  read breadcrumb and the bf16 resident-dtype message use the same
+  vocabulary.
+
+  The capability probes are unchanged, and deliberately so. They were
+  written as runtime probes rather than a version floor precisely so this
+  day would need no code change, and they still cover what a version test
+  cannot: the fixes existed for three weeks in builds that reported 0.2.1.
+  No version floor has been added to `Suggests` for the same reason: nf4
+  works on older safetensors, so a stale install costs a tier rather than
+  the model.
+
+  `recommend()`'s returned `fork_suggested` field keeps its name, which is
+  now historical: it means the installed safetensors cannot read a tier
+  the card could otherwise run. Renaming it would break the returned
+  contract for a cosmetic gain.
+
+* `reshard_safetensors()` is no longer required to make a large artifact
+  readable, since 0.3.0 fixed the overflow it worked around. It stays
+  useful for publishing: the shards it writes load on every safetensors
+  including the older ones, which is what makes a hosted artifact safe to
+  redistribute.
+
 # diffuseR 0.2.2.6
 
 * Fixed an allocator pre-warm accumulation introduced in 0.2.2.4.

R/convert_sd_pt.R

@@ -13,7 +13,7 @@
 #' CreativeML OpenRAIL++-M (redistributable), and cornball already hosts
 #' these weights as \code{.pt}. At \code{float16} the components are all
 #' sub-2 GB single files (unet ~1.7 GB, text_encoder ~0.65 GB, vae
-#' ~0.16 GB), so they load on stock CRAN safetensors.
+#' ~0.16 GB), so they also load on safetensors older than 0.3.0.
 #'
 #' @param pt_dir Directory holding \code{unet-cpu.pt},
 #'   \code{decoder-cpu.pt}, \code{text_encoder-cpu.pt} (default: the

R/download_prebuilt.R

@@ -3,8 +3,8 @@
 #' diffuseR hosts prebuilt NF4 artifacts for the two models whose
 #' licenses permit redistribution: FLUX.2-klein-4B and Z-Image-Turbo
 #' (both Apache-2.0, ungated). The artifacts are the exact output of
-#' \code{\link{flux_quantize}} (sub-2 GB shards, bf16 residents), so
-#' stock CRAN safetensors reads them. Nothing else in the catalog is
+#' \code{\link{flux_quantize}} (sub-2 GB shards, bf16 residents), so even
+#' safetensors older than 0.3.0 reads them. Nothing else in the catalog is
 #' hosted: FLUX.1-schnell sits behind a HuggingFace license gate and
 #' LTX-2.3's license does not permit redistribution, so both still
 #' download their sources and quantize locally.

R/fp8_ltx23.R

@@ -113,9 +113,11 @@ ltx23_fp8_linear <- torch::nn_module(
 #' @param output_dir Output directory for shards + manifest; NULL (the
 #'   default) resolves under tools::R_user_dir("diffuseR", "data").
 #' @param shard_bytes Numeric. Target shard size in bytes. The default
-#'   1.9e9 keeps every shard under the 2^31-byte (~2.15 GB) ceiling that
-#'   stock CRAN safetensors can read. Pass a larger value (e.g. 4e9) only
-#'   for local builds you will read back with a fork-patched safetensors.
+#'   1.9e9 keeps every shard under the 2^31-byte (~2.15 GB) offset
+#'   ceiling, so the artifact reads on a safetensors that lacks the
+#'   overflow fix (mlverse/safetensors#14, which reached CRAN in 0.3.0).
+#'   Larger values (e.g. 4e9) need that fix present, so keep them for
+#'   local artifacts and use the default for anything redistributed.
 #' @param force Logical. Re-quantize even if a valid manifest exists.
 #' @param verbose Logical.
 #'

R/nf4_ltx23.R

@@ -259,9 +259,11 @@ ltx23_nf4_linear <- torch::nn_module(
 #' @param output_dir Output directory for shards + manifest; NULL (the
 #'   default) resolves under tools::R_user_dir("diffuseR", "data").
 #' @param shard_bytes Numeric. Target shard size in bytes. The default
-#'   1.9e9 keeps every shard under the 2^31-byte (~2.15 GB) ceiling that
-#'   stock CRAN safetensors can read. Pass a larger value (e.g. 4e9) only
-#'   for local builds you will read back with a fork-patched safetensors.
+#'   1.9e9 keeps every shard under the 2^31-byte (~2.15 GB) offset
+#'   ceiling, so the artifact reads on a safetensors that lacks the
+#'   overflow fix (mlverse/safetensors#14, which reached CRAN in 0.3.0).
+#'   Larger values (e.g. 4e9) need that fix present, so keep them for
+#'   local artifacts and use the default for anything redistributed.
 #' @param force Logical. Re-quantize even if a valid manifest exists.
 #' @param verbose Logical.
 #'

R/quantize_flux.R

@@ -17,8 +17,8 @@ NULL
 }
 
 # Capability probe: can the installed safetensors round-trip this dtype?
-# CRAN safetensors (<= 0.2.1) can read bfloat16 but not write it, and has
-# no float8 support; the fixes are upstream PRs. Cached per session;
+# The CRAN safetensors 0.2.1 release can read bfloat16 but not write it,
+# and has no float8 support; 0.3.0 includes both fixes. Cached per session;
 # options(diffuseR.st_caps = list(bfloat16 = FALSE, ...)) overrides for
 # tests.
 .st_caps <- new.env(parent = emptyenv())
@@ -49,7 +49,7 @@ NULL
 # Resolve precision = "auto": prefer an existing quantized artifact
 # (fp8 first), else pick by float8 write capability. An fp8 artifact is
 # only chosen if the installed safetensors can actually read float8 -
-# otherwise a fork-built fp8 artifact on a CRAN-safetensors machine
+# otherwise an fp8 artifact on a reader without float8 support
 # would be selected and then fail at read time. Write capability is a
 # sound proxy for read capability (nothing writes fp8 but cannot read
 # it).
@@ -310,10 +310,11 @@ NULL
 #'   the per-format location under \code{tools::R_user_dir}).
 #' @param format "nf4" or "fp8".
 #' @param shard_bytes Numeric. Target shard size in bytes. The default
-#'   1.9e9 keeps every shard under the 2^31-byte (~2.15 GB) ceiling that
-#'   stock CRAN safetensors can read, so the artifact loads fork-free.
-#'   Pass a larger value (e.g. 4e9) only for local builds you will read
-#'   back with a fork-patched safetensors.
+#'   1.9e9 keeps every shard under the 2^31-byte (~2.15 GB) offset
+#'   ceiling, so the artifact reads on a safetensors that lacks the
+#'   overflow fix (mlverse/safetensors#14, which reached CRAN in 0.3.0).
+#'   Larger values (e.g. 4e9) need that fix present, so keep them for
+#'   local artifacts and use the default for anything redistributed.
 #' @param force Logical. Re-quantize even if a valid manifest exists.
 #' @param verbose Logical.
 #'
@@ -327,14 +328,15 @@ flux_quantize <- function(transformer_dir, output_dir = NULL,
     if (format == "fp8" && !.st_can_write("float8_e4m3fn")) {
         stop("The installed safetensors package cannot write float8 ",
              "tensors (needs the float8 support from ",
-             "mlverse/safetensors#13, which is merged upstream but not ",
-             "yet on CRAN; install the development version of ",
-             "safetensors from GitHub, or use format = \"nf4\").",
-             call. = FALSE)
+             "mlverse/safetensors#13, released in safetensors 0.3.0). ",
+             "Run install.packages(\"safetensors\") to update, or use ",
+             "format = \"nf4\".", call. = FALSE)
     }
     # Residents load into the compute dtype either way; bf16 halves the
-    # artifact but the CRAN build of safetensors 0.2.1 cannot write it
-    # (the fix is merged upstream; the probe decides, not the version)
+    # artifact but a safetensors without the bfloat16 write fix
+    # (mlverse/safetensors#11, which reached CRAN in 0.3.0) cannot write
+    # it. The probe decides, not the version: the fix existed for three
+    # weeks in builds still reporting 0.2.1.
     resident_dtype <- if (.st_can_write("bfloat16")) {
         torch::torch_bfloat16()
     } else {
@@ -491,10 +493,9 @@ flux_load_transformer <- function(ckpt, device = "cuda", dtype = "bfloat16",
     format <- ckpt$format %||% "full"
     if (identical(format, "fp8") && !.st_can_write("float8_e4m3fn")) {
         stop("This fp8 artifact needs float8 support the installed ",
-             "safetensors lacks (mlverse/safetensors#13, merged upstream ",
-             "but not yet on CRAN). Install the development version of ",
-             "safetensors from GitHub, or rebuild the artifact as nf4.",
-             call. = FALSE)
+             "safetensors lacks (mlverse/safetensors#13, released in ",
+             "safetensors 0.3.0). Run install.packages(\"safetensors\") ",
+             "to update, or rebuild the artifact as nf4.", call. = FALSE)
     }
     hooks <- .flux_family_hooks(ckpt$config)
 

R/quantize_gemma3.R

@@ -18,7 +18,7 @@
 #' @param output_dir Output directory for shards + manifest (default:
 #'   \code{gemma3-nf4} under \code{tools::R_user_dir}).
 #' @param shard_bytes Numeric. Target shard size in bytes; the 1.9e9
-#'   default keeps shards readable by stock CRAN safetensors.
+#'   default keeps shards readable by safetensors older than 0.3.0.
 #' @param force Logical. Re-quantize even if a valid manifest exists.
 #' @param verbose Logical.
 #'

R/recommend.R

@@ -13,7 +13,7 @@
 #'     bf16) AND the installed safetensors can \emph{read} that dtype
 #'     (\code{\link{.st_can_read}}), that tier is recommended instead.
 #'   \item When the card has room but safetensors cannot read the tier,
-#'     nf4 is recommended and the fork suggestion is surfaced in
+#'     nf4 is recommended and the upgrade suggestion is surfaced in
 #'     \code{note} (never an error).
 #' }
 #'
@@ -53,8 +53,12 @@
 #'   logical), \code{max_pixels}, \code{text_device}, \code{attn_chunk},
 #'   \code{vram_gb}, \code{pin} (page-lock the phase-swapped host
 #'   copies), \code{pinned_set_gb} (estimated pinned bytes),
-#'   \code{host_ram_gb}, \code{fork_suggested} (logical), and
-#'   \code{note} (the fork suggestion string, or NULL).
+#'   \code{host_ram_gb}, \code{fork_suggested} (logical: the installed
+#'   safetensors cannot read the tier the card could otherwise run), and
+#'   \code{note} (the suggestion string, or NULL). The field name is
+#'   historical -- the remedy used to be the cornball-ai fork and is now
+#'   \code{install.packages("safetensors")}, since the fixes reached CRAN
+#'   in 0.3.0 -- and is kept because it is part of the returned contract.
 #'
 #' @export
 #'
@@ -65,7 +69,7 @@
 #'                st_caps = list(bfloat16 = TRUE, float8_e4m3fn = FALSE))
 #' r$precision       # "nf4": fp8 fits the card, but cannot be read
 #' r$fork_suggested  # TRUE
-#' cat(r$note)       # the fork-or-nf4 message
+#' cat(r$note)       # the update-or-nf4 message
 #'
 #' # Same card, once safetensors can read float8
 #' recommend("flux1", vram_gb = 16,
@@ -137,7 +141,8 @@ recommend <- function(model = c("sd21", "sdxl", "flux1", "flux2", "zimage",
          host_ram_gb = host_ram_gb,
          fork_suggested = fork,
          note = if (fork) {
-            .st_fork_note(want$precision)
+            # The tier was blocked by .st_can_read, so diagnose read.
+            .st_update_note(want$precision, mode = "read")
         } else {
             .bf16_note(model, chosen$precision)
         }

R/reshard.R

@@ -2,13 +2,18 @@
 #'
 #' Splits a single \code{.safetensors} file into diffusers-style shards
 #' (\code{<base>-00001-of-000NN.safetensors} plus a
-#' \code{<base>.safetensors.index.json} weight map) so it loads on stock
-#' CRAN safetensors, which overflows a 32-bit offset on any file at or
-#' above 2^31 bytes. Reading the oversize source requires a
-#' fork-patched safetensors (a build-machine step); the shards it writes
-#' are fork-free to read. Used to host large fp16 diffusers weights
+#' \code{<base>.safetensors.index.json} weight map) so it loads on a
+#' safetensors without the overflow fix, which trips a 32-bit offset on
+#' any file at or above 2^31 bytes. Used to host large fp16 diffusers weights
 #' (e.g. the 5 GB SDXL UNet) unchanged, without quantization.
 #'
+#' safetensors 0.3.0 fixed the overflow (mlverse/safetensors#14), so
+#' reading an oversize source no longer needs a patched build and this is
+#' no longer required to produce a readable artifact. It stays useful for
+#' two reasons: the shards it writes load on every safetensors including
+#' the older ones, which is what makes a hosted artifact safe to publish,
+#' and sharded weights stream rather than materialising one multi-GB file.
+#'
 #' @param input Path to the source \code{.safetensors} file, or a
 #'   directory containing \code{<base>.safetensors}.
 #' @param output_dir Output directory for the shards + index.

R/resident.R

@@ -386,7 +386,7 @@ resident_load <- function(model = c("flux2", "flux1", "zimage", "ltx",
         held <- tryCatch({
             s <- torch::cuda_memory_stats(device = .cuda_index(device))
             as.numeric(s$reserved_bytes$all$current) -
-                as.numeric(s$allocated_bytes$all$current)
+            as.numeric(s$allocated_bytes$all$current)
         }, error = function(e) 0)
     }
     if (!isTRUE(is.finite(held)) || held < 0) {