format: XML in UTF-16, and a size range that stops failing on sizes a format cannot write

[donislawdev]

Two changes, and the second was found while measuring the first.

XML can be written in UTF-16

encoding (utf-8, utf-16le, utf-16be) and bom, through the textenc
package txt and md already stand on. Both default to what these documents
have always been.

XML is the first format here whose file says which encoding it is in, so it
is the first that can disagree with itself. A document announcing UTF-8 while
holding UTF-16 is the classic parser trap, and it stays the right size, hashes
stably and parses under the encoding it really is.

Measured before any of it was written, seven cases with the expected answer
written down first:

case	expat
utf-8 declaring UTF-8 (positive control)	accepts
utf-16 with a mark, declaring UTF-16	accepts
utf-16 bytes declaring UTF-8	refuses
utf-8 bytes declaring UTF-16	refuses
utf-16 without a mark, declaring UTF-16	accepts
utf-16 cut by one byte	refuses

So there is a witness outside this project, in both directions.

The fifth row decided the shape of the setting. The XML specification requires a
mark on a UTF-16 entity, and expat accepts a document without one and reads it
correctly - checked by comparing the content, not just the verdict. Nothing
here could go red on such a file, so utf-16 with bom=false is refused rather
than written.

Costs, both stated in the refusals: a UTF-16 file always has an even number of
bytes, so half of every size is unreachable and an odd size is refused naming
the nearest above and below. The smallest XML file grows from 264 B to 532 B.

A size range stops failing on sizes a format cannot write

Some formats cannot produce every byte count. A UTF-16 file is always even. A
picture cannot use the eleven byte counts just above its encoded size, because
the smallest padding chunk costs twelve, and the three OPC formats have the same
shape. A size drawn onto one of those took the whole run down, and whether
that happened depended on the seed and the number of files.

A range asks for some size between two ends rather than for a number, so a
drawn size is now moved to the nearest writable one inside the range.

The comment in drawSizes had said since 2026-09-06 that closing this "needs the
format to declare its unreachable bands, which is a change to format.Descriptor
and the owner's call". It needed no such thing - a refusal already names the
next writable size, so probing and jumping does it with no new surface. PNG
answers 143 B with 154 B, and a wide encoding answers an odd size with the next
even one.

Two things are deliberately unchanged, and the first version of this got the
line wrong until an existing guard said so:

• a range starting below what the format can do at all is still refused,
  naming the floor. zip 2000-9000 briefly produced four identical files on the
  floor instead - that is a recipe worth correcting, not a run worth filling.
• --size 1001 still refuses, because that one named a number.

Snapping is not silent: a file whose size moved carries size_moved in the
manifest, and the run says so once. The note text carries no number on purpose,
since notes group by text and a number would give one line per file.

What holds

• Ten pinned hashes unchanged for XML, whether a recipe says nothing or says
  utf-8 out loud - two different routes through the parser.
• Ten range runs across ten formats come out with the same exit code and the
  same bytes as before.
• Seven new guards, thirteen new mutations, all caught. Nine existing mutations
  retargeted after drawSizes moved to its own file - staleness.py was the
  only thing that noticed.
• engine.go went past the size ceiling, so drawSizes was cut out rather than
  the ceiling raised.

Analysis written before the code, with what was not measured:
docs/XML-ENCODING-2026-09-08.md. Verdict for the range: O190.

🤖 Generated with Claude Code