Release v0.43.1 - #11456
Draft
lidel wants to merge 18 commits into
Draft
Conversation
lidel
commented
Sep 8, 2026
lidel
commented
Contributor
- Release 0.43 #11298
* feat(cli): add --human and --sort-size to ipfs ls
- --human (-H): SI human-readable sizes in text output (humanize.Bytes)
- --sort-size (-S): sort directory entries by size, largest first
- Validation: --sort-size + --stream and --sort-size + --size=false errors
- Unit tests for formatSize and sort helpers
- CLI integration tests for both flags
- Changelog highlight in v0.44
* test(ls): make sort tests fail when sorting breaks
The tests covering --sort-size could not detect a broken feature. The
unit tests copied the comparator into the test body and sorted with their
own copy, so they passed regardless of what ls.go did. The CLI tests named
each fixture after its size, which left alphabetical order and size order
in agreement, so every ordering subtest passed even with --sort-size
disabled outright.
- extract lsLinkByName and lsLinkBySize so the tests exercise shipped code
- point the unit tests at those two functions
- name fixtures so their alphabetical order disagrees with their sizes
- pin the real directory behaviour: UnixFS directories carry no Filesize,
so they sort as 0, tie with empty files and break by name rather than
landing strictly last
* fix(ls): return 400 not 500 for bad flag combos
Passing --sort-size together with --stream or --size=false made
/api/v0/ls answer 500 Internal Server Error, telling API clients the
server had broken when the caller had simply combined flags that cannot
work together. Clients that retry on 5xx would retry a request that can
never succeed.
cmds.ErrClient maps to 400, matching how the rest of core/commands
reports caller mistakes. CLI output is unchanged.
* docs: fix --human size examples and JSON claims
The help for --human advertised "1K 234M 2G", which no kubo command has
ever printed. All three use humanize.Bytes, so the real output is SI with
a space: 1.2 kB, 234 MB, 2.0 GB. The stale example was copied into
'ipfs ls' from the two commands that already carried it, so correct all
three together.
- ls, repo stat, bitswap stat: examples now match real output
- drop the claim that --enc=json reports bytes. On the CLI, 'ipfs ls'
has a PostRun that prints the text table whatever --enc says, so no
JSON is produced there at all. Only the /api/v0/ls response is JSON,
and that part is true
- directories have no UnixFS Filesize, so they sort as 0 and tie with
empty files. They do not land strictly last, so stop saying they do
- changelog: a #### highlight with a TOC entry, kept short and pointing
at 'ipfs ls --help' for the details
- format sizes with strconv.FormatUint rather than fmt.Sprintf("%d")
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
(cherry picked from commit d4bbbef)
Signed-off-by: weifanglab <weifanglab@outlook.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> (cherry picked from commit 9d5290e)
The key expired in 2018 and the pgp.mit.edu lookup it pointed at returns 503, so anyone following it hits a dead end twice. No successor key is published for security@ipfs.io on any keyserver, so the notes point at the repository security policy instead. Seeding the init docs changes their directory CID, so the constants the tests assert against move with it. (cherry picked from commit 48012cb)
* docs: guide for running on low-memory devices Add docs/production/low-memory.md with tuning for 8 GiB devices like Raspberry Pi: GOMEMLIMIT, systemd MemoryHigh/MemoryMax, and Provide.DHT.MaxWorkers, plus how to verify with cgroup memory pressure. - README.md, docs/README.md: link the guide - docs/config.md: link from lowpower profile and ResourceMgr notes, caution against setting Swarm.ResourceMgr.MaxMemory too low - misc/systemd: commented example limits pointing at the guide * docs: announcement sizing and dht client advice Extend the low-memory guide with measured announcement rates, the 48h record expiry floor, and a worked 10M-CID example (MaxWorkers=6, Interval=32h). Advise client-only routing (autoclient), disabling the AutoNAT and relay services, and warn against enabling the accelerated DHT client on constrained hardware. - docs/environment-variables.md: document GOMEMLIMIT - docs/config.md: broaden lowpower profile pointer to the guide (cherry picked from commit bb0ac77)
* ci: remove self-hosted runners * test: fix zsh completion test on hosted runners The test runs compinit in a non-interactive zsh. When a directory in fpath is writable by group or others, compinit asks a question. There is no terminal to answer it, so compinit aborts and the test fails. GitHub-hosted runner images have such a directory. compinit -i skips insecure directories without asking. The test checks kubo's generated completion script, not the permissions of the host's zsh directories. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org> (cherry picked from commit c374658)
…11428) * fix(key): restrict overwritten key exports to owner-only permissions Signed-off-by: questfever <questfever@outlook.com> * fix(atomicfile): temp file leak and name limit Both problems surface through `ipfs key export`, which now writes through this helper, but they affect every caller: config writes, repo migrations and `ipfs update`. - remove the temporary file when the rename fails, so one holding private key material is not left next to the target - keep the ".tmp-" prefix and the random suffix within the 255 byte file name limit, so a target with a long name can still be written * fix(key): route key export by target type Choosing the write path with os.Lstat treated /dev/stdout, /dev/stderr and /dev/fd/N as plain paths, because they are symlinks into /proc/self/fd, so the atomic write failed on targets that worked before. Decide by what the path resolves to, and state the whole contract in the command help. - regular file or nothing yet: written to a temporary file and renamed over the target, following symlinks, including one whose target does not exist yet - character device or pipe: streamed in place, confirmed on the open descriptor and without O_TRUNC, so a path swapped for a regular file can neither receive the key nor be emptied - anything else: refused, naming the path - errors name the file the user asked for, and the temporary file is flushed before the rename * fix(key): stop export landing on the wrong file An export could replace a file the target symlink does not point at. resolveSymlink joined a relative link target onto the path as typed, so ".." collapsed lexically. Where a parent component was itself a symlink, the join named a file outside the directory the link resolves to: the key was renamed over that file, and the intended target was never written. The link's parent is now resolved with filepath.EvalSymlinks before the join. * test(key): drop umask dependency in export test os.WriteFile applies the umask, so under umask 077 the fixture was created 0600 and the check that a failed export leaves the file at 0644 failed for reasons unrelated to the code under test. The CLI test already chmods for the same reason. --------- Signed-off-by: questfever <questfever@outlook.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> (cherry picked from commit 842d9aa)
(cherry picked from commit de88597)
* fix(fuse): report a link count of 1 st_nlink was left at 0, which POSIX gives an inode with no remaining names, so tools can read a live file as one on its way out. Neither IPFS nor MFS has hard links. Directories report 1 as well, which keeps GNU find from trusting a subdirectory count and skipping entries. * fix(ipns): fill attrs in key directory lookups The /ipns root answers lookups for its key directories and alias symlinks itself, and the reply carried zeroed attributes. Every later lookup refreshed the kernel's cache with the same zeroes, so the Getattr that would have corrected them never ran and a key directory showed up as d--------- with no link count. * fix(fuse): give mounts stable inode numbers go-fuse numbers any node left with a zero StableAttr.Ino itself, and picks a new number every time. The kernel drops a mount's dentries once EntryTimeout expires, so a file nobody touched came back from the next lookup under a different st_ino, and programs that compare file identity over time read that as the file being replaced. vim abandons a save with "E949: File changed while writing", which is what made the FUSE CI job flaky once it moved to slower runners and the save started crossing the one second cache boundary. - /ipfs takes the number from the multihash digest inside the CID, so the same content is one object whichever path reaches it; inline CIDs hash the whole CID instead, their digest being the content itself - /ipns and /mfs allocate per mount from a counter keyed by parent and name, retired on unlink, rmdir and rename so a name that is created again is never handed a removed entry's number - writable nodes carry a generation of their own, so go-fuse builds a fresh node per lookup instead of reusing one bound to an *mfs.File that boxo has since replaced - mount points report inode 1 instead of 0, and readdir reports the same numbers as stat * fix(fuse): tell two CIDs apart on /ipfs go-fuse matches a lookup against the nodes it already holds by the whole of StableAttr, so two entries that agree on it are served as one object. The inode number alone is 63 bits, and two CIDs can end up sharing one, by chance or by choice. Reading the second one then returned the first one's bytes. A dag-pb CID and the raw CID of the same block hit this every time, because the number ignored the codec. The number and the generation now both come from a hash of the codec and the multihash, so it takes a match on 128 bits to confuse two entries. The mount also sets FirstAutomaticIno instead of relying on go-fuse's default, so the range it keeps for itself stays where the code says it is. * fix(fuse): stat /ipfs entries we cannot read A stat of a child whose block is missing, or of one in a codec this mount does not decode, read UnixFS metadata that was never loaded and panicked. go-fuse does not recover a panic in its serve loop, so this took the whole daemon down. Neither case is exotic: a dag-cbor object linked from a UnixFS directory needs no missing blocks at all. A lookup that cannot read the block now fails instead of building an entry from it, and a block that is not UnixFS is reported as a file of its own size. * fix(fuse): keep a file a rename cannot move `mv /ipns/<key>/f /ipns/f` unlinked the source before finding out that the /ipns root holds no files of its own, then failed with EINVAL and left the file nowhere. The destination is now checked before anything is written. * fix(fuse): store a moved file where it landed A rename across directories only wrote the source directory back, so the file's new name lived in memory until something else flushed it, and a daemon that stopped first lost the file. The destination is written back first, so an interrupted rename leaves the file under both names rather than under neither. * fix(fuse): keep the inode number over a rename Both names gave up their inode numbers on rename, so a moved file came back about a second later as a different file, which is the problem this branch fixes everywhere else. A moved directory was worse: its entries are keyed by its number, so the whole subtree was renumbered and the old keys were left behind until unmount. The number now moves with the entry. Nothing else has to change to make that safe: go-fuse cannot hand back the old node anyway, because every node gets a generation of its own. The comments that credited the renumbering for it were wrong. * test(fuse): check listings and stat agree Each mount fills in the inode number of a directory entry separately from the one it reports to stat, and nothing compared the two. Tools read whichever is cheaper for them. * docs(config): warn about writing to a mounted mfs `ipfs files` writes to the same tree as the /mfs and /ipns mounts without the mount knowing, so the two can lose each other's writes. (cherry picked from commit 93868b9)
* fix(fuse): keep a rename's writes go-fuse hands the kernel's existing node to the new name once Dir.Rename returns, and that node still held the MFS handle the rename had unlinked. MFS treats such a handle as gone: a write through it was accepted and then dropped, so `mv a b` followed by a write to `b` read back the old contents a second later, once the entry cache expired. A directory was worse. Creating a file in one that had just been renamed flushed through the dead handle, which carried the name the rename had moved away from, so the new file was lost and the old directory came back for good. Each node now reaches its MFS handle through an atomic, and a rename points the moved node at the entry that exists afterwards. Entries the kernel had already looked up underneath a renamed directory hang off the handle it was reached through, so the walk follows them down; it covers what the kernel is holding, not the whole tree. Invalidating the entry instead was tried and does not work: the kernel processes FUSE_NOTIFY_INVAL_ENTRY while holding the parent inode lock, so notifying from inside Rename deadlocks, and notifying asynchronously still loses most of the writes it races. Left unfixed: a write through a file descriptor held open across the rename still goes to the descriptor opened from the old handle. * fix(fuse): refuse to replace a non-empty directory A rename may only overwrite a directory that is empty. MFS removes a directory and everything under it without complaint, so `mv -T src dst` took dst's contents with it and reported success. Rmdir already had the check; Rename now makes it too, before it unlinks anything. The check for an absent destination also goes through errors.Is now. It compares against a sentinel that boxo returns bare today, and the cost of that changing is the source file, which by then has been unlinked. * fix(fuse): read /ipfs blocks we cannot decode A UnixFS directory can link to a block of any codec. stat reports one the mount cannot decode as a file the size of the block, but every read of it failed, because the read path went looking for a UnixFS DAG that is not there. A size stat promises has to be a size reads deliver, so serve the block itself. * fix(fuse): list a directory with a missing block One child whose block is not held locally failed the whole listing, and with an errno the caller could make nothing of: ipld.ErrNotFound has no mapping, so it arrived as ENOSYS. The readable entries are worth having, so report the one that is missing with no type and let a stat of it say what is wrong. * fix(fuse): report the CID the path used The ipfs.cid xattr answered with a CID the caller had never seen. A lookup rebuilds the node by decoding the block, which drops the version and codec of the path it came from, so a v1 dag-pb path reported its v0 form. Keep the CID the entry resolved to and report that. The changelog entry also covers the rename check from the commit before it, which landed without one. * test(fuse): make the rename tests catch their bugs TestRenameOntoNamespaceRoot read the file back through the mount, which answers from the entry the kernel still has cached and so succeeds whether or not the rename took the file away. It passed against the bug it was written for. Ask MFS instead. The dirent helper also loops on a record length it never checks, which would spin rather than fail if the kernel ever sent zero. * test(ipns): settle the repo path before mounting TestStatfs assigned Root.RepoPath once the server was already serving, and Statfs reads it from a FUSE handler goroutine, so `go test -race ./fuse/...` reported a data race on every run. The mfs and readonly tests already settle it before their mount; do the same here. (cherry picked from commit a73e8c0)
* feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 (cherry picked from commit a554311)
(cherry picked from commit f9aad61)
* chore: pin unreleased go-libp2p-kad-dht Validation branch for libp2p/go-libp2p-kad-dht#1293 (batched provider datastore writes): pins its head commit so kubo CI runs against it before it merges. Not for merging; repoint at a tagged release once one exists. - go.mod (x3): go-libp2p-kad-dht v0.42.1 => 5a1c2420 pseudo-version; pion/stun v3.1.5 rides along as a transitive bump from kad-dht master - core/node/provider.go: the pin pulls in libp2p/go-libp2p-kad-dht#1288, whose WithAddLocalRecord callback now receives the provide system's lifecycle context; use it instead of context.Background() * chore: bump kad-dht to v0.42.2 * docs: changelog entry --------- Co-authored-by: guillaumemichel <guillaume@michel.id> (cherry picked from commit 329838a)
* feat: opt-in unixfs-v1-2026 profile (IPIP-550) Opt-in Data-first PBNode field ordering via the new unixfs-v1-2026 config profile, per IPIP-550. Defaults and the preexisting unixfs-v0-2015 and unixfs-v1-2025 profiles are unchanged and keep their CIDs. - config: Import.UnixFSPBNodeFieldOrder (links-first default, data-first) and the unixfs-v1-2026 profile applying unixfs-v1-2025 settings plus data-first - core/node: wires merkledag.DefaultPBNodeFieldOrder from config - deps: boxo bump to the ipfs/boxo#1212 encoder commit - test/cli: byte-exact fixtures from the IPIP-550 table; pinned CIDs for existing profiles unchanged Refs ipfs/specs#550 * ci: run gateway-conformance from ipip-550 commit Temporary pin to the ipfs/gateway-conformance#304 head so the PBNode field ordering tests run against kubo. Switch back to a tagged release once one ships. * ci: gateway-conformance back to v0.14 v0.14.1 shipped the ipfs/gateway-conformance#304 tests, so the moving v0.14 tag covers them again. * docs: unixfs-v1-2026 scope and MFS re-encode State what the profile actually changes: every dag-pb node with both Data and Links gets a new CID, files larger than one chunk included, and data already in MFS is upgraded to the new order on first read (plain `ipfs files ls` or `stat` included), a sharded root before its child shards; the MFS root is re-encoded by any command that starts a node. - config/profile.go, docs/config.md: profile description and the unixfs-v1-2026 section - docs/changelogs/v0.43.md: highlight leads with the need (readers get the HAMT layout before links) and the upgrade-on-first-use behavior * chore: update boxo to 04a079ec27b1 Pins the ipfs/boxo#1212 branch tip: data-first bytes derive from dagpb.AppendEncode so link sorting is inherited, unknown field order values return an error, and pinned end-to-end CIDs cover the profile. * refactor: drop unixfs-v1-2026, keep opt-in knob A dated successor profile invites unintentional adoption and a de facto new CIDv1 default. Import.UnixFSPBNodeFieldOrder stays as the documented low-level opt-in; unixfs-v0-2015 and unixfs-v1-2025 now pin links-first explicitly. - deps: boxo bump to the ipfs/boxo#1212 commit that drops UnixFS_v1_2026 - test/cli: field order exercised via the config knob, same IPIP-550 fixture bytes and CIDs Refs ipfs/specs#550 * chore: update boxo to 02026ddcf262 Squash-merge of ipfs/boxo#1212 on main. * docs: tighten v0.43.1 changelog - tighter CID profile entry; MFS caveat now says re-encode happens on rewrite, not read - deps: boxo pseudo-version with ipfs/boxo#1212, gateway-conformance v0.14.1 * docs: tighten Import.UnixFSPBNodeFieldOrder docs (cherry picked from commit 9dbaab7)
(cherry picked from commit 94be4c2)
* fix: empty Bootstrap list no longer dials stale backup peers Rework of #11453 per maintainer review (lidel): the fix belongs in boxo (bootstrap.bootstrapRound skips the backup list when no bootstrap peers are configured), not behind a Routing.Type=none guard in kubo. This PR is now the companion to ipfs/boxo#1213: - Pin the boxo fix via a temporary replace directive pointing at the boxo PR branch. Once ipfs/boxo#1213 merges, repoint at boxo main and convert to a pseudo-version pin. - Add a kubo-level regression test (TestBootstrapWithEmptyPeerListAndStaleBackupPeers) verifying IpfsNode.Bootstrap runs with an empty Bootstrap config and a populated TempBootstrapPeersKey without error. - Document under Bootstrap in docs/config.md that an empty list disables all bootstrap dialing, including saved backup peers. - Add a v0.44 changelog highlight. Drop the v0.43 entry and the broken emoji from the original PR. Closes #11452 Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore: review follow-ups for empty Bootstrap list Replace the kubo-level test with one that fails without the boxo fix (a mocknet backup peer must not be dialed when Bootstrap is empty), shorten the changelog and config.md text, and point the boxo replace at the reviewed commit. * test: bootstrap fallback needs a configured peer TestBackupBootstrapPeers used an empty Bootstrap list to force the fallback to saved peers. An empty list now disables bootstrap dialing entirely, so the test configures an unreachable bootstrap peer instead. The changelog and config.md describe what still connects with an empty list and when backup peers are used. * chore: update boxo to 2edf737db3aa Drops the temporary replace on a fork now that ipfs/boxo#1213 is merged, and moves the changelog entry to v0.43.1. --------- Co-authored-by: kalou <research@beleganjur.kalou.net> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> (cherry picked from commit a13ea47)
maybeRunGC ran PeriodicGC on req.Context, which the RPC shutdown command never cancels (it only closes the node), so gcErrc never closed and daemonFunc blocked in merge() forever. Cancel the GC context on node close as well, keeping req.Context so a signal still aborts an in-flight sweep before the OnStop hooks close the datastore. Closes #11424 (cherry picked from commit ceafeba)
* docs: spell out datastore layouts in profiles Profile names and docs now say which store holds what: 'flatfs-levelds' (blocks in flatfs, everything else in leveldb) is the canonical name of the default layout, with 'flatfs' and 'flatfs-measure' kept as aliases. 'flatfs-pebbleds' and its '-measure' twin name the flatfs plus pebble layout that otherwise needs a hand-written Datastore.Spec. - docs: why flatfs holds only blocks, measure overhead note, Datastore.Spec example, profile sections - cmd/ipfs: datastore paragraph in 'ipfs init --help' - config: spec functions named after the profiles they back - test/cli: layout on disk, restart, aliases, default, measure metrics - test/sharness: profiles added to t0025 Closes #11423 * fix: safer flatfs shardFunc setup at init The flatfs shard depth can only be set when a repo is created, by passing a config file to 'ipfs init', and that route had traps. - init: refuse a config without Identity.PrivKey ('ipfs config show' output) on both 'ipfs init <file>' and 'ipfs daemon --init', and validate Datastore.Spec before writing anything - config profile apply: refuse profiles that change the on-disk layout - fsrepo: mismatch error labels config and datastore_spec correctly and says the layout is fixed at init - docs: why and when to use next-to-last/3, the config-file procedure, migration by moving data to a new repo * docs: add go-ds-pebble v0.5.13 to v0.43.1 changelog (cherry picked from commit 088e79f)
The collector at telemetry.ipshipyard.dev shuts down with the end of Shipyard's IPFS work. The built-in endpoint is now empty, so a node collects nothing and sends nothing unless its operator configures one, and it drops a telemetry_uuid left by an earlier version.
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.