Point safetensors messaging at CRAN 0.3.0 instead of a GitHub build - #65
Open
TroyHernandez wants to merge 5 commits into
Open
Point safetensors messaging at CRAN 0.3.0 instead of a GitHub build#65TroyHernandez wants to merge 5 commits into
TroyHernandez wants to merge 5 commits into
Conversation
safetensors 0.3.0 reached CRAN on 2026-08-21 with all four fixes diffuseR had been routing users around: float8 (mlverse/safetensors#13), bfloat16 write (#11), the >2 GB offset overflow (#14), and empty tensor names (#10). Every message telling users to install a development build from GitHub was, as of that date, advising the harder path to a fix already sitting in `install.packages("safetensors")`. Five messages carried it: recommend()'s tier note and the graceful fp8/bf16 fallback (both via the shared helper), flux_quantize()'s two errors, and the >2 GB read breadcrumb. Plus README's tier table and the st_caps/reshard docs. THE PROBES ARE UNCHANGED, and that is the point. They were written as runtime capability probes rather than a version floor precisely so this day would need no code change, and they earned it: a 0.3.0 user got the higher tiers the moment they updated, with nothing in the package to adjust. They also still cover what a version test cannot -- the fixes existed for three weeks in builds reporting 0.2.1, so the version number never distinguished them. No floor added to Suggests for the same reason: nf4 works on older safetensors, so a stale install costs a tier, not the model. TESTS NOW PIN THE REMEDY, NOT JUST THE PHRASING. The existing assertions checked "best fit for your card", the absence of an em dash, and the file size in the breadcrumb -- everything except what the message tells a user to DO. That is how the advice went stale silently and would have stayed stale: nothing failed. Added guards that the messages name install.packages and do NOT mention a development version or GitHub, and checked the guards reject the old wording rather than merely passing on the new. .st_fork_note is renamed .st_update_note, since it no longer suggests a fork. Four call sites, all internal. recommend()'s returned `fork_suggested` field KEEPS its name. It is part of a documented return contract that memory_flux.R propagates and five tests read, so renaming it would break callers for a cosmetic gain. The docs now say what it means and that the name is historical. reshard_safetensors() is no longer required to make a large artifact readable. It stays useful for publishing: its shards load on every safetensors including older ones, which is what makes a hosted artifact safe to redistribute. NEWS records this as a new 0.2.2.7 entry rather than editing 0.2.2.3's rationale, which described the situation accurately when it was written. Suite 1220 assertions, 0 failures.
The first pass fixed the messages and left the documentation, which is
where most of the stale advice actually lived. The shard_bytes help for
flux_quantize(), ltx23_quantize_nf4(), ltx23_quantize_fp8() and
gemma3_quantize_nf4() still described the 1.9e9 default as what "stock
CRAN safetensors" can read, and told users to install a fork for
anything larger. README, the performance-levers vignette, and the
unet_safetensors / download_prebuilt / convert_sd21_pt_to_diffusers
pages carried variants of the same. All now describe the floor as
"older than 0.3.0" and name install.packages("safetensors") as the
remedy.
st_caps.R contradicted itself: the opening said safetensors 0.2.1 lacks
the fixes, while the explanation below correctly noted that fixed
development builds also reported 0.2.1. The opening now says the CRAN
0.2.1 release, which is what was meant, and the probe-don't-pin
rationale survives intact.
The two direct fp8 errors had their wording corrected last commit but
nothing asserted it: both tests only matched "float8", so the old
GitHub-development-build advice would have passed. Both gates now assert
the exact CRAN remedy and reject "GitHub" and "development version".
Checked that these guards are not vacuous by running all four
pre-change messages through them: every one fails all three assertions.
The load-path fixture needs the class. flux_load_transformer() runs
stopifnot(inherits(ckpt, "ltx23_checkpoint")) before the fp8 gate, so a
bare list(format = "fp8") errors on the stopifnot and never reaches the
message under test.
Suite 1229 assertions, 0 failures. man/ regenerates byte-identical from
source. Historical NEWS entries left alone.
recommend() gates tiers on .st_can_read(), but the note it produced for a blocked bf16 tier cited mlverse/safetensors#11, which is the bfloat16 WRITE fix. bfloat16 read worked on CRAN 0.2.1, so a reader that lacks it is not waiting on #11: the message was well-formed, carried the right remedy, and pointed at an issue unrelated to the user's failure. .st_update_note() now takes mode, and both call sites pass the capability they gated on: recommend() passes "read", .st_graceful_precision() forwards its own mode. The read path for bf16 drops the issue reference entirely, because no release added bfloat16 read and there is no fix to point at. float8 is untouched in both modes, since 0.2.1 had neither read nor write for it. The suite was holding this in place rather than catching it: its read-mode assertion matched "safetensors#11", so the wrong reference was pinned by the test. That assertion now matches the read wording. The remedy-only guards from the previous commit could not have caught this class of defect, which is the point of the added assertions: checked that both new ones fail against the old output while the remedy guard still passes it. Also stopped making the version the requirement in prose. The shard_bytes docs for flux_quantize(), ltx23_quantize_nf4(), ltx23_quantize_fp8(), plus reshard_safetensors() and the README fp8 bullet, said "requires safetensors 0.3.0 or newer". That contradicts the probe-don't-pin rationale in the same PR, which turns on capable builds having existed while reporting 0.2.1. They now name the fix and note where it landed. Suite 1242 assertions, 0 failures, re-run after the rformat pass against a fresh install. man/ regenerated from source.
Four sites still described the missing capability as a version. The worst contradicted itself in adjacent sentences: flux_quantize()'s resident-dtype comment said "safetensors before 0.3.0 cannot write it. The probe decides, not the version: the fix existed for three weeks in builds still reporting 0.2.1." Both halves cannot be true. The others: the >2 GB read breadcrumb told users "safetensors before 0.3.0 overflows", unet_safetensors said "on safetensors older than 0.3.0", and .st_read_or_breadcrumb's comment used "safetensors 0.3.0+" as shorthand for a reader with the fix. All now name the capability and say where it landed. NEWS was narrating the iteration rather than the result: one paragraph said the docs now require 0.3.0 or newer, the next said that wording was replaced because it was wrong. Collapsed to the end state. Left alone deliberately: convert_sd_pt, quantize_gemma3 and download_prebuilt say sub-2 GB artifacts load on readers older than 0.3.0, which is a compatibility claim that holds for every build, capable or not. README says "releases before 0.3.0 overflow", which is accurate as scoped, since no release before 0.3.0 carried the fix. Suite 1242 assertions, 0 failures. The overflow breadcrumb still carries 2^31, the size, the shard name and the CRAN remedy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
safetensors 0.3.0 reached CRAN on 2026-08-21 with all four fixes diffuseR had been routing users around: float8 (mlverse/safetensors#13), bfloat16 write (#11), the >2 GB offset overflow (#14), and empty tensor names (#10).
As of that date every message in the package advising a GitHub development build was pointing at the harder path to a fix already sitting in
install.packages("safetensors").What changed
Five user-facing messages carried the stale advice:
recommend()'s tier note and the graceful fp8/bf16 fallback (shared helper)flux_quantize()float8 errorsPlus README's tier table and the
st_caps/reshard_safetensorsdocs.What deliberately did not change
The capability probes. They were written as runtime probes rather than a version floor precisely so this day would need no code change — and they earned it: a 0.3.0 user got the higher tiers the moment they updated, with nothing in the package to adjust. They also still cover what a version test cannot, since the fixes existed for three weeks in builds reporting
0.2.1.No version floor in
Suggests, for the same reason. nf4 works on older safetensors, so a stale install costs a tier rather than the model.recommend()'sfork_suggestedfield keeps its name. It is part of a documented return contract thatmemory_flux.Rpropagates and five tests read; renaming it would break callers for a cosmetic gain. The docs now state what it means and that the name is historical.The tests were the actual gap
The existing assertions checked
"best fit for your card", the absence of an em dash, and the file size in the breadcrumb — everything except what the message tells a user to do. That is how the advice went stale silently and would have stayed stale: nothing failed.Added guards that the messages name
install.packagesand do not mention a development version or GitHub, and verified the guards reject the old wording rather than merely passing on the new.Note on NEWS
Recorded as a new 0.2.2.7 entry rather than editing 0.2.2.3's rationale. That entry described the situation accurately when it was written; rewriting it would be revising history rather than recording a change.
Suite 1220 assertions, 0 failures. The first commit is a one-line rformat carryover from #64, kept separate.