Skip to content

# UDF padded its directories and could read only one block size - #197

Merged
Hawkynt merged 1 commit into
mainfrom
fix/udf-both-directions
Sep 6, 2026
Merged

# UDF padded its directories and could read only one block size#197
Hawkynt merged 1 commit into
mainfrom
fix/udf-both-directions

Conversation

@Hawkynt

@Hawkynt Hawkynt commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Our writer produced volumes the kernel could not walk, and our reader threw on every image mkudffs built. The two agreed with each other, which is why nothing caught it.

Write side

Directories were padded onto block boundaries. WriteFidBlockAligned zero-padded whenever a File Identifier Descriptor would straddle a block, on the stated premise that ECMA-167 §4/14.4 forbids the crossing. It does not, and mkudffs does it — in a volume it built, the record at byte 2020 runs into the next block. Linux reads directory bytes as one dense run, so it stopped at our pad byte:

udf_verify_fi: directory (ino 259) has entry at pos 2020 with incorrect tag 0

repeated without end; the walk never returned. Every directory over 2048 bytes was unreadable, and udfinfo says nothing about it.

dstrings were bare ASCII — no compression byte, no used-length byte at PVD+55 (ECMA-167 §1/7.2.12) — so udfinfo reported label, lvid, vid, vsid and fsid as empty.

Six required structures were missing: no reserve descriptor sequence, one anchor where §3/8.4 asks for two, no Unallocated Space Descriptor, no Implementation Use Volume Descriptor, no Logical Volume Integrity Descriptor, no integrity extent in the LVD. udfinfo printed seven warnings and called the volume inconsistent.

Also: an empty file was given a block and every extent was rounded up to a sector, so the allocation claimed more than the information length; FID FileVersionNumber and tag location were left at 0 where OSTA UDF §2.3.4.1 requires 1, as were LogicalBlocksRecorded, Checkpoint, timestamps and unique identifiers; names were UTF-8 under compression identifier 8, which means one byte per character (§2.1.1); and files past 1 GiB could not be represented at all.

Read side

The anchor's address was scaled by 2048, and the partition start too. Logical block 256 is counted in blocks, so every mkudffs volume at 512, 1024 or 4096 threw UDF: invalid AVDP tag before a byte was read.

Compression identifier 8 decoded as UTF-8 (same fault in four more files) — café.bin came back as caf\uFFFD.bin.

Allocation-descriptor continuation was refused. Once an object's extents outgrow its File Entry the rest live in an Allocation Extent Descriptor (§4/14.5). At 512 bytes a block, a 200-entry directory needs one: 300 of 332 files vanished.

Evidence

  • Ours: udfinfo exits 0 with empty stderr (was seven warnings), label=CWBUDF integrity=closed udfrev=2.01. mount -o loop,ro succeeds and all 577 files of the size sweep come back byte for byte, driver log clean. 7z t — "Everything is Ok", 577 files / 9 folders.
  • Theirs: mkudffs at 512/1024/2048/4096 and revisions 1.02/1.50/2.00/2.01, filled through the kernel — zero differing lines. Likewise a 64 MB volume of 579 files including an 8 MiB file and a 500-entry directory.
  • Multi-extent: a 1,073,745,920-byte file takes two short ADs (1073739776 + 6144, the first a whole-block multiple as §2.3.10.1 requires), mounts, and the kernel's sha256 matches ours.
  • In-place edit and the space wiper stay native-clean: after 40 adds, an empty add and a remove, udfinfo is silent; wiping 203 dirtied free blocks zeroes exactly 415,744 bytes and leaves the reserve sequence, integrity descriptor and both anchors intact.

xorriso reads neither ours nor mkudffs's — it wants an ISO 9660 volume, so it is not a verdict either way.

Revert checks

reverted result
CS0 back to UTF-8 native read-back fails at all 4 block sizes
block-size probe → 2048 only fails at 512, 1024, 4096
continuation not followed fails at 512
FID padding restored kernel emits incorrect tag 0 and the walk hangs — recorded as a hung run, not a clean assertion
integrity descriptor dropped udfinfo acceptance fails with the three LVID warnings

Tests

UdfNativeToolTests (6 cases) and UdfOnDiskStructureTests (6 tool-free structural cases). UDF joins FileSizeBoundaryTests.

114 passed / 1 skipped for ~Udf. Collateral surfaces complete and green: 6391 docs/descriptor/schema/registry, 293 driver/interop/mounting, 4545 constraints/operations/layout/streaming — 0 failures.

Fence report

Five test files changed because their assertions encoded the old behaviour: the PD sector moved 33→34 as the VDS gained descriptors; NSR02NSR03 since the writer now records the UDF 2.01 its descriptor always claimed; the wipe test's extent-length premise; the driver test's continuation expectation, renamed to BrokenContinuationAllocationDescriptorFailsMountClosed since valid continuations are now read; and the synthetic fixture's anchor, which needed a real tag checksum.

Noticed, not fixed:

  • UdfExtentMap, UdfModifier and UdfBlockMover still hardcode 2048-byte blocks and an anchor at block 256, so defrag, in-place edit and the block mover only work on volumes our own writer produced. Listing and extraction are block-size agnostic.
  • Udf_OurImage_UdffsckAccepts can never run (udftools 2.3 ships no udffsck), and Udf_LinuxMkudffsOutput_ReadByOurReader only asserts entries != null on an empty mkudffs volume — a test that cannot fail, and exactly the self-consistency trap. Both are superseded; deleting them is your call.
  • The writer declares MinTotalArchiveSize = 1 MB but a minimal volume is 538,624 bytes.
  • UdfFormatDescriptor.Create drops explicitly-supplied empty directories.

@Hawkynt
Hawkynt force-pushed the fix/udf-both-directions branch from df5c43d to b57814d Compare September 6, 2026 15:22
The writer and the reader agreed with each other and with nothing else. Every
fault below was found by putting foreign software on one side: mkudffs and the
kernel's udf driver built and filled the volumes we read, udfinfo and that same
driver read the volumes we wrote.

Writing:

- A File Identifier Descriptor may span a logical block boundary — ECMA-167
  4/14.4, and mkudffs does it: in a volume it built, the record at byte 2020 of
  a directory runs into the next block. We padded the boundary instead, so the
  driver hit a zero tag where the next record belongs: "udf_verify_fi:
  directory (ino 259) has entry at pos 2020 with incorrect tag 0", repeated
  without end, and the walk never returned. Every directory over 2048 bytes was
  unreadable.
- Volume identifiers were written as bare ASCII. ECMA-167 1/7.2.12 spells a
  dstring as a compression byte, the characters, and the used length in the
  field's last byte, so udfinfo reported the label, the logical volume, the
  volume set and the file set as empty.
- The volume carried one anchor, one descriptor sequence, and neither an
  unallocated space, implementation use nor integrity descriptor. udfinfo
  printed seven warnings and called the logical volume inconsistent. It now
  records both sequences, both anchors, and a closed integrity descriptor whose
  file and directory counts match the tree.
- File names went out as UTF-8 under compression identifier 8, which means one
  byte per character. Anything above U+007F was unreadable to everything else.
- An empty file was still given a block, and a file's extent was rounded up to
  one, so the allocation said more than the size did. Extents now stop at the
  logical length and a zero-length file records none.
- File version number, tag locations, blocks recorded, checkpoint and unique
  identifiers were all left at zero. Files past 1 GiB now take several extents;
  past what one File Entry can hold, CanAccept declines rather than the writer
  producing a volume nothing can address.

Reading:

- The anchor's address, logical block 256, is counted in blocks, and we assumed
  2048. Every volume mkudffs formats at 512, 1024 or 4096 threw "invalid AVDP
  tag" before a byte was read. The block size is now probed from the anchor and
  confirmed against the logical volume descriptor.
- Compression identifier 8 was decoded as UTF-8; it is one byte per character.
  A file the kernel called cafe.bin came back with a replacement character.
- Allocation descriptors stopped at the first continuation (extent type 3), so
  an object whose extents outgrow its File Entry lost everything past it. At
  512 bytes a block that is a directory of 200 entries: 300 of 332 files
  vanished. Continuations are now followed, with a visited set bounding the walk.

In place editing keeps up with the rest: both partition descriptors grow, the
second anchor follows the partition, and the integrity descriptor's size, free
space and file count are maintained.

Verified: a volume with the size sweep, long and Unicode names, nested
directories and a 500-entry directory mounts read-only and all 577 files come
back byte for byte, with nothing in the driver log; udfinfo exits zero with an
empty stderr and reports the right label, revision, counts and closed
integrity; 7z tests it clean. In the other direction, volumes mkudffs built at
512, 1024, 2048 and 4096 bytes a block and the kernel filled read back byte for
byte, as does a 64 MB volume of 579 files. Each fix was reverted in turn and
the matching test fails without it.
@Hawkynt
Hawkynt force-pushed the fix/udf-both-directions branch from b57814d to b06d0a5 Compare September 6, 2026 15:56
@Hawkynt
Hawkynt merged commit b61f3b9 into main Sep 6, 2026
4 checks passed
@Hawkynt
Hawkynt deleted the fix/udf-both-directions branch September 6, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant