# UDF padded its directories and could read only one block size - #197
Merged
Conversation
Hawkynt
force-pushed
the
fix/udf-both-directions
branch
from
September 6, 2026 15:22
df5c43d to
b57814d
Compare
The writer and the reader agreed with each other and with nothing else. Every fault below was found by putting foreign software on one side: mkudffs and the kernel's udf driver built and filled the volumes we read, udfinfo and that same driver read the volumes we wrote. Writing: - A File Identifier Descriptor may span a logical block boundary — ECMA-167 4/14.4, and mkudffs does it: in a volume it built, the record at byte 2020 of a directory runs into the next block. We padded the boundary instead, so the driver hit a zero tag where the next record belongs: "udf_verify_fi: directory (ino 259) has entry at pos 2020 with incorrect tag 0", repeated without end, and the walk never returned. Every directory over 2048 bytes was unreadable. - Volume identifiers were written as bare ASCII. ECMA-167 1/7.2.12 spells a dstring as a compression byte, the characters, and the used length in the field's last byte, so udfinfo reported the label, the logical volume, the volume set and the file set as empty. - The volume carried one anchor, one descriptor sequence, and neither an unallocated space, implementation use nor integrity descriptor. udfinfo printed seven warnings and called the logical volume inconsistent. It now records both sequences, both anchors, and a closed integrity descriptor whose file and directory counts match the tree. - File names went out as UTF-8 under compression identifier 8, which means one byte per character. Anything above U+007F was unreadable to everything else. - An empty file was still given a block, and a file's extent was rounded up to one, so the allocation said more than the size did. Extents now stop at the logical length and a zero-length file records none. - File version number, tag locations, blocks recorded, checkpoint and unique identifiers were all left at zero. Files past 1 GiB now take several extents; past what one File Entry can hold, CanAccept declines rather than the writer producing a volume nothing can address. Reading: - The anchor's address, logical block 256, is counted in blocks, and we assumed 2048. Every volume mkudffs formats at 512, 1024 or 4096 threw "invalid AVDP tag" before a byte was read. The block size is now probed from the anchor and confirmed against the logical volume descriptor. - Compression identifier 8 was decoded as UTF-8; it is one byte per character. A file the kernel called cafe.bin came back with a replacement character. - Allocation descriptors stopped at the first continuation (extent type 3), so an object whose extents outgrow its File Entry lost everything past it. At 512 bytes a block that is a directory of 200 entries: 300 of 332 files vanished. Continuations are now followed, with a visited set bounding the walk. In place editing keeps up with the rest: both partition descriptors grow, the second anchor follows the partition, and the integrity descriptor's size, free space and file count are maintained. Verified: a volume with the size sweep, long and Unicode names, nested directories and a 500-entry directory mounts read-only and all 577 files come back byte for byte, with nothing in the driver log; udfinfo exits zero with an empty stderr and reports the right label, revision, counts and closed integrity; 7z tests it clean. In the other direction, volumes mkudffs built at 512, 1024, 2048 and 4096 bytes a block and the kernel filled read back byte for byte, as does a 64 MB volume of 579 files. Each fix was reverted in turn and the matching test fails without it.
Hawkynt
force-pushed
the
fix/udf-both-directions
branch
from
September 6, 2026 15:56
b57814d to
b06d0a5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Our writer produced volumes the kernel could not walk, and our reader threw on every image
mkudffsbuilt. The two agreed with each other, which is why nothing caught it.Write side
Directories were padded onto block boundaries.
WriteFidBlockAlignedzero-padded whenever a File Identifier Descriptor would straddle a block, on the stated premise that ECMA-167 §4/14.4 forbids the crossing. It does not, and mkudffs does it — in a volume it built, the record at byte 2020 runs into the next block. Linux reads directory bytes as one dense run, so it stopped at our pad byte:repeated without end; the walk never returned. Every directory over 2048 bytes was unreadable, and
udfinfosays nothing about it.dstrings were bare ASCII — no compression byte, no used-length byte at PVD+55 (ECMA-167 §1/7.2.12) — so
udfinforeported label, lvid, vid, vsid and fsid as empty.Six required structures were missing: no reserve descriptor sequence, one anchor where §3/8.4 asks for two, no Unallocated Space Descriptor, no Implementation Use Volume Descriptor, no Logical Volume Integrity Descriptor, no integrity extent in the LVD.
udfinfoprinted seven warnings and called the volume inconsistent.Also: an empty file was given a block and every extent was rounded up to a sector, so the allocation claimed more than the information length; FID
FileVersionNumberand tag location were left at 0 where OSTA UDF §2.3.4.1 requires 1, as were LogicalBlocksRecorded, Checkpoint, timestamps and unique identifiers; names were UTF-8 under compression identifier 8, which means one byte per character (§2.1.1); and files past 1 GiB could not be represented at all.Read side
The anchor's address was scaled by 2048, and the partition start too. Logical block 256 is counted in blocks, so every mkudffs volume at 512, 1024 or 4096 threw
UDF: invalid AVDP tagbefore a byte was read.Compression identifier 8 decoded as UTF-8 (same fault in four more files) —
café.bincame back ascaf\uFFFD.bin.Allocation-descriptor continuation was refused. Once an object's extents outgrow its File Entry the rest live in an Allocation Extent Descriptor (§4/14.5). At 512 bytes a block, a 200-entry directory needs one: 300 of 332 files vanished.
Evidence
udfinfoexits 0 with empty stderr (was seven warnings),label=CWBUDF integrity=closed udfrev=2.01.mount -o loop,rosucceeds and all 577 files of the size sweep come back byte for byte, driver log clean.7z t— "Everything is Ok", 577 files / 9 folders.mkudffsat 512/1024/2048/4096 and revisions 1.02/1.50/2.00/2.01, filled through the kernel — zero differing lines. Likewise a 64 MB volume of 579 files including an 8 MiB file and a 500-entry directory.udfinfois silent; wiping 203 dirtied free blocks zeroes exactly 415,744 bytes and leaves the reserve sequence, integrity descriptor and both anchors intact.xorrisoreads neither ours nor mkudffs's — it wants an ISO 9660 volume, so it is not a verdict either way.Revert checks
incorrect tag 0and the walk hangs — recorded as a hung run, not a clean assertionudfinfoacceptance fails with the three LVID warningsTests
UdfNativeToolTests(6 cases) andUdfOnDiskStructureTests(6 tool-free structural cases). UDF joinsFileSizeBoundaryTests.114 passed / 1 skipped for
~Udf. Collateral surfaces complete and green: 6391 docs/descriptor/schema/registry, 293 driver/interop/mounting, 4545 constraints/operations/layout/streaming — 0 failures.Fence report
Five test files changed because their assertions encoded the old behaviour: the PD sector moved 33→34 as the VDS gained descriptors;
NSR02→NSR03since the writer now records the UDF 2.01 its descriptor always claimed; the wipe test's extent-length premise; the driver test's continuation expectation, renamed toBrokenContinuationAllocationDescriptorFailsMountClosedsince valid continuations are now read; and the synthetic fixture's anchor, which needed a real tag checksum.Noticed, not fixed:
UdfExtentMap,UdfModifierandUdfBlockMoverstill hardcode 2048-byte blocks and an anchor at block 256, so defrag, in-place edit and the block mover only work on volumes our own writer produced. Listing and extraction are block-size agnostic.Udf_OurImage_UdffsckAcceptscan never run (udftools 2.3 ships noudffsck), andUdf_LinuxMkudffsOutput_ReadByOurReaderonly assertsentries != nullon an empty mkudffs volume — a test that cannot fail, and exactly the self-consistency trap. Both are superseded; deleting them is your call.MinTotalArchiveSize = 1 MBbut a minimal volume is 538,624 bytes.UdfFormatDescriptor.Createdrops explicitly-supplied empty directories.