Expect a refusal where the soak may build one

[beetlebugorg]

What

An operation reports whether the module may refuse a decodable source given the argument the generator produced. A chain holding one allows either result, so a refusal counts as a result rather than a failure.

MODE=bench drives only requests a decodable source must answer with an image, so a run measures throughput and every failure in it is a real one.

run.sh sets DIMS_ENABLE_DIMS3=on.

Why

The soak reported a 3.4% failure rate against an unchanged module. Measured on main at 2c69660 with seed 1788290612: 483 failures in 14,801 responses. Every one had the same shape, expect: image with tamper: none and a 400 response, and the module's reasons were The crop region lies outside the image, Parsing thumbnail geometry failed, and Quality must be a number from 1 to 100.

The generator treats a chain that parses as valid. The module refuses a chain it cannot carry out. Those agreed when the harness landed, and #113 changed the module without changing the generator. A real regression producing a few dozen bad responses cannot be seen through a standing 3.4%.

The manifest records no source dimensions, so the generator cannot tell whether a crop it built falls inside the image. It reports what it knows instead: this chain holds a crop, so either result is allowed.

Building the safe profile found two more faults.

/dims3/ returned 404 for every request. #122 turned that endpoint off in the image, and the generator sends /dims3/. In bench mode that was 13.6% of all requests, and in soak mode it hid inside the refusals the run already tolerated.

A resize with ^ or < enlarges, and a chain that enlarges then converts can pass the dimension limit of the output format. format/webp/resize/62x847!/resize/1298x778^ asks WebP for an image 17,700 pixels tall, and WebP stops at 16383. The safe profile resizes with a form that shrinks or fits.

Verify

MODE=bench DURATION=300 CONNECTIONS=8 bash test/endurance/run.sh
MODE=soak  DURATION=180 CONNECTIONS=16 bash test/endurance/run.sh

I measured 0 failures in 8,611 bench requests at 28.7 requests per second, and 0 failures in 4,543 mixed requests. Both reported no child crash.

A dump shows the two profiles apart:

dims_soak --manifest m.tsv --safe --dump 6
dims_soak --manifest m.tsv --dump 400

Every safe request reports none and image. In the mixed profile no chain holding a crop or a thumbnail reports image.

Breaking

A report from an earlier run counts refusals the new one does not. The failure count is comparable only across runs of the same version.