From fac145f9e64a9d425fadd2ad7eacf7e865756bec Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Wed, 9 Sep 2026 20:13:30 +0200 Subject: [PATCH] damage: one rule about accept, two callers, and a site that lists every command Three defects that were all the same shape: a list written out by hand that nothing compared against its source. The refusal for a target that damages its files and declares they will be accepted lived in the recipe reader alone. The command line never skipped past it - it never reached it, so `--damage zero-head --expected accept` ended with code 0 and wrote a manifest saying a deliberately broken file should be accepted, while the identical recipe was refused with code 3. The condition is now `Chain.ConflictsWithExpectation` in internal/damage with two CALLERS rather than two copies: the recipe reader, which keeps reporting it beside the other problems of that recipe and with the address of the target, and the engine, which every surface passes through. The command line ends with code 2 and writes nothing. Only accept is refused - reject, sanitize and unspecified beside damage, and accept without damage, all still work, and the guard asserts that so this cannot become a wall. The website listed nine of the ten commands, because `tfg damage` arrived and the list was a hand copy of what `tfg --help` prints. site.Facts carried no commands at all, so nothing could compare. The block is rendered now: names from the help itself, summaries from the language file, both directions checked, and the parser refuses rather than returning an empty list - a guard that stopped finding the block would compare the page against nothing and pass. The site also gains a section on producing files that are broken on purpose, and README gains the --damage row its flag table never had. sign_release.py asked the certificate store through the Cert: drive, which does not exist when Windows PowerShell 5.1 is launched from pwsh. The error is non terminating, so the exit code was zero, and stderr was read only on a non-zero code - the script announced a missing card while the card was in the reader. It opens the store through .NET now and prints what PowerShell said whatever the code was. Guards: three new, eight mutations, all caught. Two came back NOT CAUGHT first and both were statements about the mutation rather than the code. Co-Authored-By: Claude Opus 5 --- .github/scripts/sign_release.py | 56 ++++++++++++++--- CHANGELOG.md | 27 ++++++++ README.md | 6 ++ internal/cli/errors.go | 13 ++++ internal/damage/refusals.go | 46 ++++++++++++++ internal/engine/damage.go | 36 +++++++++++ internal/engine/engine.go | 2 +- internal/guard/damagerefused_test.go | 89 +++++++++++++++++++++++++++ internal/guard/signing_test.go | 42 +++++++++++++ internal/guard/site_test.go | 78 ++++++++++++++++++++++- internal/recipe/damage.go | 10 ++- internal/site/site.go | 53 ++++++++++++---- internal/site/view.go | 30 +++++++++ web/content/en/docs.html | 55 ++++++++++++++--- web/content/en/site.json | 12 ++++ web/content/pl/docs.html | 56 ++++++++++++++--- web/content/pl/site.json | 12 ++++ web/public/docs/index.html | 46 ++++++++++++++ web/public/pl/dokumentacja/index.html | 47 ++++++++++++++ web/templates/partials.html | 5 ++ 20 files changed, 677 insertions(+), 44 deletions(-) diff --git a/.github/scripts/sign_release.py b/.github/scripts/sign_release.py index 71d2cc4..6f333c2 100644 --- a/.github/scripts/sign_release.py +++ b/.github/scripts/sign_release.py @@ -114,6 +114,22 @@ def powershell(script): capture_output=True, text=True) if out.returncode != 0: raise SystemExit("sign_release: powershell failed:\n%s" % out.stderr.strip()) + # PowerShell errors are NON TERMINATING by default, so a script can print + # a page of complaints and still exit zero. Reading stderr only on a + # non-zero code therefore threw away the one sentence that said what went + # wrong, and left the caller looking at empty output with no reason for it. + # + # It cost an hour on 2026-09-09 signing v0.3.0: the certificate lookup came + # back empty and the script blamed a missing card, while the card was in + # the reader and readable. The complaint was there the whole time and + # nothing printed it. O200. + # + # A note rather than a failure, because a warning is not a refusal and the + # caller may have asked something that legitimately produces one. + if out.stderr.strip(): + print(" powershell also said:") + for line in out.stderr.strip().splitlines(): + print(" %s" % line) return out.stdout @@ -170,17 +186,41 @@ def signing_thumbprint(pin): the only selector it takes, and the repository pins SHA-256 because that is the digest worth pinning. Resolving one to the other here means the two can never drift apart in a configuration file. + + THE STORE IS OPENED THROUGH .NET RATHER THAN THROUGH THE Cert: DRIVE, and + that is a measurement rather than a preference. The drive is provided by + Microsoft.PowerShell.Security, which Windows PowerShell 5.1 only loads when + PSModulePath points at its own module directory - and a 5.1 launched from + inside pwsh 7 is handed pwsh's PSModulePath instead. Measured 2026-09-09 on + this machine, from a python started under pwsh: + + Get-ChildItem Cert:\\CurrentUser\\My -> 0, plus + "Cannot find drive. A drive with the name 'Cert' does not exist." + X509Store('My','CurrentUser') -> 11 + + Both ended with code ZERO, because a PowerShell error is non terminating - + so the script saw empty output and reported a missing card while the card + was in the reader. It cost an hour signing v0.3.0 and the release went out + through Git Bash as a workaround. X509Store is in the runtime rather than + in a module, so it does not depend on which shell started which. O200. """ script = ( "$out = @(); " - "Get-ChildItem Cert:\\CurrentUser\\My, Cert:\\LocalMachine\\My " - "-ErrorAction SilentlyContinue | Where-Object { " - " $_.Extensions.EnhancedKeyUsages.Value -contains '%s' } | ForEach-Object { " - " $h = [System.Security.Cryptography.SHA256]::Create().ComputeHash($_.RawData); " - " $out += [pscustomobject]@{ " - " sha256 = (($h | ForEach-Object { $_.ToString('x2') }) -join ''); " - " thumb = $_.Thumbprint; subject = $_.Subject; " - " notAfter = $_.NotAfter.ToString('s') } " + "foreach ($where in 'CurrentUser', 'LocalMachine') { " + " $store = New-Object System.Security.Cryptography.X509Certificates.X509Store('My', $where); " + " try { $store.Open('ReadOnly') } catch { continue }; " + " foreach ($c in $store.Certificates) { " + " $eku = @(); " + " foreach ($x in $c.Extensions) { " + " if ($x -is [System.Security.Cryptography.X509Certificates.X509EnhancedKeyUsageExtension]) { " + " foreach ($u in $x.EnhancedKeyUsages) { $eku += $u.Value } } }; " + " if ($eku -notcontains '%s') { continue }; " + " $h = [System.Security.Cryptography.SHA256]::Create().ComputeHash($c.RawData); " + " $out += [pscustomobject]@{ " + " sha256 = (($h | ForEach-Object { $_.ToString('x2') }) -join ''); " + " thumb = $c.Thumbprint; subject = $c.Subject; " + " notAfter = $c.NotAfter.ToString('s') } }; " + " $store.Close() " "}; $out | ConvertTo-Json -Compress" % CODE_SIGNING_OID ) entries = json.loads(powershell(script).strip() or "[]") diff --git a/CHANGELOG.md b/CHANGELOG.md index a4244e3..2cb0148 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,33 @@ because it turns other people's test suites red. ## [Unreleased] +### Fixed + +- **Asking for damaged files and declaring they will be accepted is now refused + on the command line too.** A damaged file is one a reader was measured to + refuse, so `--expected accept` beside `--damage` asks for something nothing + can deliver. + + A recipe saying the same thing has always been refused. The command line was + not: it wrote the files and recorded in the manifest that a deliberately + broken file should be accepted, which is the one place this tool must not say + something untrue. `tfg generate --damage zero-head --expected accept` now + ends with exit code `2` and writes nothing, and a recipe still ends with `3` + and names the target the problem is in. + + Only `accept` is refused. `reject` is what damage already means, and + `sanitize` and `unspecified` are both real questions to ask about a broken + file - a system under test may be meant to repair it, or that may be the + point of the test - so all three still work, as does `--expected accept` on + files that are not damaged. + +- **The documentation website lists every command the tool has.** `tfg damage` + arrived in 0.3.0 and the page describing the commands still showed the other + nine, because that list was written out by hand. The page now takes the list + from the program itself, so a command added later cannot go missing from it, + and the site has a section explaining how to produce a file that is broken on + purpose. + ## [0.3.0] - 2026-09-09 ### Breaking diff --git a/README.md b/README.md index f85e991..ab4882e 100644 --- a/README.md +++ b/README.md @@ -262,6 +262,7 @@ tfg generate --format txt --size 1mb settings come from the flags | `--out ` | directory to write into. Default `.` | | `--seed ` | run seed. The same seed gives the same bytes | | `--set =` | a format setting, repeatable: `--set width=1920 --set height=1080` | +| `--damage ` | break the files on purpose, repeatable and applied in order. Run `tfg damage` for the list | | `--expected ` | `accept`, `reject`, `sanitize` or `unspecified` | | `--expected-reason ` | why that outcome, from the closed list below | | `--preset ` | build the set a named test question calls for | @@ -350,6 +351,11 @@ The smallest file a damage can be given follows its settings, so the column is measured with the defaults. Ask for less and the run is refused before anything is written, naming a size that would work. +Asking for `--expected accept` beside a damage is refused too, because nothing +could meet it. Write `sanitize` if the system under test is meant to repair the +file, or `unspecified` if that is the question you are asking - both of those, +and `reject`, work as they always did. + ## 馃摐 Recipes A recipe is a YAML file describing a whole run. Commit it beside your tests and diff --git a/internal/cli/errors.go b/internal/cli/errors.go index a0c8eb4..2058e0e 100644 --- a/internal/cli/errors.go +++ b/internal/cli/errors.go @@ -217,6 +217,19 @@ func classifyRequest(err error) (int, bool) { if errors.As(err, &tooSmall) { return ExitFormat, true } + // Damaging a file and declaring it will be accepted is two flags that + // cancel each other, which is a fault in the invocation rather than a + // request no format can meet - the same conflict stands for all of them. + // A script reading ExitFormat goes looking for another format or another + // size, and neither of those is the fix. Owner's call on 2026-09-09. + // + // Anything arriving here came off the command line: a recipe declaring the + // same pair is refused while the recipe is read, with the address of the + // target and code 3 beside its other problems. O199. + var impossible *damage.ExpectationConflictError + if errors.As(err, &impossible) { + return ExitUsage, true + } if code, ok := classifyFormat(err); ok { return code, true } diff --git a/internal/damage/refusals.go b/internal/damage/refusals.go index 9f9c732..f569080 100644 --- a/internal/damage/refusals.go +++ b/internal/damage/refusals.go @@ -114,6 +114,52 @@ func (e *NoChangeError) Error() string { return e.What() + ". " + e.Why() } +// RuledOutExpectation is the one declared outcome a damaged file cannot have. +// +// It is spelled here rather than imported because internal/manifest sits +// beside this package rather than under it, so the two cannot see each other. +// TestTheOutcomeDamageRulesOutIsTheOneTheManifestKnows compares this against +// manifest.OutcomeAccept and against the list a recipe accepts, which is what +// stops three spellings of one word from drifting apart. +const RuledOutExpectation = "accept" + +// ConflictsWithExpectation is the refusal a target earns by damaging its files +// and declaring they will be accepted, or nil when there is no conflict. +// +// It lives here, on the chain, because it is a fact about damage rather than +// about either surface - and both surfaces ask it. Measured on 2026-09-09 with +// the check living in the recipe reader alone: a recipe was refused with code +// 3 while the identical run off the command line ended with code 0 and wrote a +// manifest saying a deliberately broken file should be accepted. The recipe +// reader still asks first, so it keeps reporting this beside every other +// problem of that recipe and with the address of the target - what changed is +// that the engine asks too, so no surface can get past it. See O199. +// Two shapes of one rule, and the pair is deliberate. The engine wants an +// error to hand upwards, while the recipe reader wants the parts - What, Why +// and Instead - to lay out beside the other problems of that recipe. +// +// Written as two functions rather than one returning the concrete type, +// because that one would be the typed nil trap: a nil *ExpectationConflictError +// placed in an error interface is NOT a nil error. Measured 2026-09-09 on a +// four case program - "reject", "sanitize", "" and "accept" all came back +// err != nil - so the engine would have refused EVERY target, damaged or not, +// and the guard beside this one asserts exactly that it does not. +func (c Chain) ConflictsWithExpectation(expected string) error { + if bad := c.ExpectationConflict(expected); bad != nil { + return bad + } + return nil +} + +// ExpectationConflict is the same question answered with the refusal itself, +// or nil. For a caller that needs the parts rather than an error. +func (c Chain) ExpectationConflict(expected string) *ExpectationConflictError { + if len(c) == 0 || expected != RuledOutExpectation { + return nil + } + return &ExpectationConflictError{Outcome: expected} +} + // ExpectationConflictError is a target that damages a file and expects it to // be accepted. // diff --git a/internal/engine/damage.go b/internal/engine/damage.go index 6b3ea96..f7d25c8 100644 --- a/internal/engine/damage.go +++ b/internal/engine/damage.go @@ -13,6 +13,24 @@ import ( "github.com/donislawdev/TestingFilesGenerator/internal/manifest" ) +// checkDamage is every refusal a damaged target can earn during planning. +// +// One entry rather than two calls side by side in engine.go, and that is the +// same measurement this file was cut out for: adding the second call put +// engine.go at 411 lines of code against a ceiling of 408, and the answer to a +// ceiling is a cut rather than a larger number. These two belong together +// anyway - both are "what makes this target impossible before a byte is +// written", which is one subject. +// +// The floor first, because it names a number a person can act on. A target +// earning both refusals gets that one. +func checkDamage(t *Target) error { + if err := checkDamageFloor(t); err != nil { + return err + } + return checkDamageExpectation(t) +} + // checkDamageFloor refuses a file smaller than the damage it was given. // // Here rather than at the moment of writing, and that is the point: a file @@ -46,6 +64,24 @@ func checkDamageFloor(t *Target) error { return nil } +// checkDamageExpectation refuses a target that breaks its files and declares +// they will be accepted. +// +// Here as well as in the recipe reader, and that is the whole point of it. The +// condition is one function on the chain, so this is a second CALLER rather +// than a second copy - what it buys is that the command line reaches it, and +// the command line never reads a recipe. Measured on 2026-09-09 before this +// existed: the recipe was refused with code 3 while +// --damage zero-head --expected accept ended with code 0 and wrote a manifest +// claiming a deliberately broken file should be accepted. O199. +// +// The window cannot reach this today - damage sits on the generate screen and +// the expectation on the recipe screen - and being here rather than in the +// reader is what covers it on the day those two meet. +func checkDamageExpectation(t *Target) error { + return t.Damage.ConflictsWithExpectation(t.Expected) +} + // damageFor records what was broken about this file, with the settings // resolved rather than as written. // diff --git a/internal/engine/engine.go b/internal/engine/engine.go index 9604571..3734183 100644 --- a/internal/engine/engine.go +++ b/internal/engine/engine.go @@ -278,7 +278,7 @@ func settleTarget(t *Target, opt Options, seen map[string]bool) (format.Descript // After the draw, because a range arrives here carrying only a count and a // damage floor has to be judged against the sizes that will really be // written. - if err := checkDamageFloor(t); err != nil { + if err := checkDamage(t); err != nil { return format.Descriptor{}, err } return desc, nil diff --git a/internal/guard/damagerefused_test.go b/internal/guard/damagerefused_test.go index 02382b4..5150dc9 100644 --- a/internal/guard/damagerefused_test.go +++ b/internal/guard/damagerefused_test.go @@ -1,12 +1,16 @@ package guard import ( + "bytes" + "context" "errors" "os" "path/filepath" + "slices" "strings" "testing" + "github.com/donislawdev/TestingFilesGenerator/internal/cli" "github.com/donislawdev/TestingFilesGenerator/internal/damage" "github.com/donislawdev/TestingFilesGenerator/internal/engine" "github.com/donislawdev/TestingFilesGenerator/internal/format" @@ -313,6 +317,91 @@ targets: } } +// The command line refuses that pair as well, and writes nothing. +// +// The guard above asks recipe.Parse and only recipe.Parse, and that was enough +// to be green through a build where this was broken. Measured on 2026-09-09 on +// the 0.3.0 binary: the recipe was refused with code 3 while +// --damage zero-head --expected accept ended with code 0 and left a manifest +// on disk saying a deliberately damaged file should be accepted - the tool +// lying in the one place its value lives. O199. +// +// Both halves are needed. Asking only the refusal would pass for a build that +// refuses damage beside any expectation at all, and asking only that files are +// written would pass for one that refuses nothing - so the second loop is the +// wall detector and the first is the hole detector. +// +// The count of files is asked rather than the exit code alone: a refusal that +// arrives after the writing has started is a refusal that came too late, and +// the code by itself cannot tell those apart. +func TestTheCommandLineRefusesDamageBesideAcceptToo(t *testing.T) { + run := func(t *testing.T, extra ...string) (int, string, int) { + t.Helper() + dir := t.TempDir() + var out, errOut bytes.Buffer + args := append([]string{ + "generate", "--format", "txt", "--size", "100", + "--damage", damage.ZeroHead, "--out", dir, + }, extra...) + code := cli.Run(context.Background(), args, &out, &errOut) + written, err := os.ReadDir(dir) + if err != nil { + t.Fatalf("reading the output directory: %v", err) + } + return code, errOut.String(), len(written) + } + + code, said, files := run(t, "--expected", damage.RuledOutExpectation) + if code != cli.ExitUsage { + t.Errorf("damage beside %q ended with %d, expected %d - two flags that cancel each other are a fault in the invocation\nstderr: %s", + damage.RuledOutExpectation, code, cli.ExitUsage, said) + } + if files != 0 { + t.Errorf("the run was refused and still left %d file(s) behind", files) + } + if !strings.Contains(said, damage.RuledOutExpectation) { + t.Errorf("the refusal does not name the word it turned down: %s", said) + } + + // The other three are legitimate questions about a broken file, and a + // build refusing them would be a wall rather than this rule. + for _, outcome := range []string{"reject", "sanitize", "unspecified"} { + if code, said, _ := run(t, "--expected", outcome); code != cli.ExitOK { + t.Errorf("--expected %s beside damage ended with %d rather than %d: %s", + outcome, code, cli.ExitOK, said) + } + } + // And the expectation on its own is untouched. Every case above carries a + // damage, so a build that refused accept for every run whatsoever would + // look correct from all of them. + dir := t.TempDir() + var out, errOut bytes.Buffer + if code := cli.Run(context.Background(), []string{ + "generate", "--format", "txt", "--size", "100", + "--expected", damage.RuledOutExpectation, "--out", dir, + }, &out, &errOut); code != cli.ExitOK { + t.Errorf("--expected %s with nothing damaged ended with %d rather than %d, which makes this a wall rather than a rule about damage: %s", + damage.RuledOutExpectation, code, cli.ExitOK, errOut.String()) + } +} + +// The outcome damage rules out is the one the manifest and the recipe know. +// +// Three spellings of one word live in three packages that cannot import each +// other - damage sits beside manifest rather than under it - so this compares +// them rather than leaving them to drift. A rename in one place turns this red +// instead of quietly producing a build where nothing is ever refused. +func TestTheOutcomeDamageRulesOutIsTheOneTheManifestKnows(t *testing.T) { + if damage.RuledOutExpectation != manifest.OutcomeAccept { + t.Errorf("damage rules out %q and the manifest calls it %q, so nothing would ever match", + damage.RuledOutExpectation, manifest.OutcomeAccept) + } + if !slices.Contains(recipe.Outcomes(), damage.RuledOutExpectation) { + t.Errorf("damage rules out %q and a recipe does not accept that word at all: %v", + damage.RuledOutExpectation, recipe.Outcomes()) + } +} + // A damage the build does not know is refused while the recipe is read, and // the refusal names what there is. // diff --git a/internal/guard/signing_test.go b/internal/guard/signing_test.go index d9e592a..4c295fc 100644 --- a/internal/guard/signing_test.go +++ b/internal/guard/signing_test.go @@ -119,6 +119,48 @@ func TestTheSigningScriptRefusesBeforeItSigns(t *testing.T) { } } +// The signing script asks the certificate store in a way that does not depend +// on which shell started it, and never hides what PowerShell said. +// +// Both halves are one measurement from 2026-09-09, signing v0.3.0. Windows +// PowerShell 5.1 launched from inside pwsh 7 is handed pwsh's PSModulePath, so +// Microsoft.PowerShell.Security is not loaded and the Cert: drive it provides +// does not exist. Measured from a python started under pwsh: +// +// Get-ChildItem Cert:\CurrentUser\My -> 0 and "Cannot find drive" +// X509Store('My','CurrentUser') -> 11 +// +// The error is NON TERMINATING, so the exit code was zero and the script - +// which read stderr only on a non-zero code - saw empty output and announced a +// missing card while the card was in the reader. The release went out through +// Git Bash as a workaround and the hour spent on it is O200. +// +// Neither half is enough alone. Reading stderr without changing the lookup +// leaves a script that explains its own failure every time, and changing the +// lookup without reading stderr leaves the next non terminating error just as +// invisible - the message a person needs would still be thrown away. +func TestTheSigningScriptDoesNotDependOnWhichShellStartedIt(t *testing.T) { + script := signingScript(t) + + // The .NET store is in the runtime rather than in a module, so it answers + // whatever PSModulePath says. + if !strings.Contains(script, "X509Store") { + t.Error("the script does not open the certificate store through .NET, so it answers differently depending on the shell that launched it") + } + // The drive is the half that goes missing. Named as a Windows path here so + // this cannot match the .NET call above. + for _, drive := range []string{`Cert:\CurrentUser`, `Cert:\LocalMachine`} { + if strings.Contains(script, drive) { + t.Errorf("the script reads %s, a drive that does not exist when Windows PowerShell is launched from pwsh - and its absence is reported with exit code zero", drive) + } + } + // A non-zero code is not the only way PowerShell says something is wrong, + // so the check that reads stderr must not be reached only through one. + if !strings.Contains(script, "out.stderr.strip():") { + t.Error("nothing prints what PowerShell said unless the exit code is non-zero, and a non terminating error leaves that code at zero") + } +} + // The workflow that speaks about signed bytes must not claim to have built them. func TestTheAttestationWorkflowDoesNotClaimToHaveBuiltAnything(t *testing.T) { attest := workflowText(t, "attest-release.yml") diff --git a/internal/guard/site_test.go b/internal/guard/site_test.go index 7ec90b3..ed4d7a1 100644 --- a/internal/guard/site_test.go +++ b/internal/guard/site_test.go @@ -2,13 +2,16 @@ package guard import ( "bytes" + "context" "encoding/json" "encoding/xml" "errors" + "fmt" "io" "os" "path/filepath" "regexp" + "slices" "sort" "strconv" "strings" @@ -89,8 +92,62 @@ func exitCodesInOrder() []int { } } +// commandsTheToolPrints is the command names out of tfg --help, which is the +// text the page is a copy of. +// +// Asked of the help rather than of the router beside it on purpose: the help +// is what a visitor compares the page against, so agreeing with anything else +// would prove the wrong thing. +func commandsTheToolPrints(t *testing.T) []string { + t.Helper() + var out, errOut bytes.Buffer + if code := cli.Run(context.Background(), []string{"--help"}, &out, &errOut); code != cli.ExitOK { + t.Fatalf("tfg --help ended with %d rather than %d, so there is no list to compare against", + code, cli.ExitOK) + } + names, err := commandNamesIn(out.String()) + if err != nil { + t.Fatalf("reading the command block out of tfg --help: %v", err) + } + return names +} + +// commandNamesIn takes the names out of the Commands: block of a help text. +// +// Split on two spaces rather than on the first one, because "recipe fmt" is a +// command whose name has a space in it - splitting on the first would name a +// command the router does not have. +// +// Finding nothing is an error rather than an empty list, and that is the whole +// reason this is a function of its own. A guard that quietly stopped finding +// the block would compare the page against nothing and stay green while +// proving it - which is the failure this file exists to make impossible. +func commandNamesIn(help string) ([]string, error) { + const header = "Commands:" + _, rest, found := strings.Cut(help, header+"\n") + if !found { + return nil, fmt.Errorf("no %q line in the help text", header) + } + var names []string + for _, line := range strings.Split(rest, "\n") { + if !strings.HasPrefix(line, " ") { + break + } + name, _, split := strings.Cut(strings.TrimPrefix(line, " "), " ") + if !split { + return nil, fmt.Errorf("the line %q under %s has no summary beside the name", line, header) + } + names = append(names, name) + } + if len(names) == 0 { + return nil, fmt.Errorf("the %s block is empty", header) + } + return names, nil +} + // factsFromTheProgram fills every number the pages may state. -func factsFromTheProgram() site.Facts { +func factsFromTheProgram(t *testing.T) site.Facts { + t.Helper() formats := make([]site.Format, 0, len(format.All())) for _, d := range format.All() { props := make([]site.Property, 0, len(d.Properties)) @@ -132,6 +189,7 @@ func factsFromTheProgram() site.Facts { Formats: formats, ExitCodes: exitCodesInOrder(), Presets: ids, + Commands: commandsTheToolPrints(t), Downloads: declaredDownloads(), // Fixed on purpose. See the comment on the field. Year: 2026, @@ -188,7 +246,7 @@ func siteUnderTest(t *testing.T) site.Site { t.Helper() root := webRoot(t) return site.Site{ - Facts: factsFromTheProgram(), + Facts: factsFromTheProgram(t), Languages: languagesOnDisk(t), ContentDir: filepath.Join(root, "content"), TemplateDir: filepath.Join(root, "templates"), @@ -385,7 +443,7 @@ func withoutYamlComments(in string) string { } func TestEveryLanguageDescribesEverythingTheProgramCanProduce(t *testing.T) { - facts := factsFromTheProgram() + facts := factsFromTheProgram(t) for _, lang := range languagesOnDisk(t) { for _, code := range facts.ExitCodes { if _, ok := lang.Endings[strconv.Itoa(code)]; !ok { @@ -397,6 +455,20 @@ func TestEveryLanguageDescribesEverythingTheProgramCanProduce(t *testing.T) { t.Errorf("the preset %q has no question in %s", id, lang.Code) } } + // The other direction as well, which the rows above do not ask. A + // command dropped from the program leaves its summary behind in both + // language files, and the page would then be a list of what the tool + // used to have - the same defect as a missing one, read backwards. + for _, name := range facts.Commands { + if _, ok := lang.Commands[name]; !ok { + t.Errorf("the command %q has no summary in %s, so the page would list it blank", name, lang.Code) + } + } + for name := range lang.Commands { + if !slices.Contains(facts.Commands, name) { + t.Errorf("%s describes a command %q that tfg --help does not print", lang.Code, name) + } + } for _, f := range facts.Formats { // A format with no independent reader says "none" in the registry, // which is an identifier rather than a sentence. The page turns it diff --git a/internal/recipe/damage.go b/internal/recipe/damage.go index 0d20cf6..04220d1 100644 --- a/internal/recipe/damage.go +++ b/internal/recipe/damage.go @@ -166,11 +166,17 @@ func reportDamageSetting(p *problems, at spot, bad error) { // value lives. Damage winning overwrites what somebody wrote, and the // regression surface says an expectation stated in a recipe reaches the // manifest unchanged. Owner's call on 2026-09-09. +// +// The condition itself is damage's rather than this file's, since 2026-09-09. +// Asking it here as well as in the engine is what keeps this reported with the +// address of the target and beside every other problem of the same recipe - +// while the engine asking it is what covers the command line, which never +// reads a recipe at all. One rule, two callers. See O199. func refuseImpossibleExpectation(p *problems, where spot, t Target) { - if len(t.Damage) == 0 || t.Expected != outcomeAccept { + refusal := t.Damage.ExpectationConflict(t.Expected) + if refusal == nil { return } - refusal := &damage.ExpectationConflictError{Outcome: t.Expected} p.add(where.of(refusal.AboutSetting()), fmt.Sprintf("%s: %s", where, refusal.What()), refusal.Why(), refusal.Instead()) diff --git a/internal/site/site.go b/internal/site/site.go index fcd7151..2637d55 100644 --- a/internal/site/site.go +++ b/internal/site/site.go @@ -77,6 +77,26 @@ type Preset struct { Question string } +// Command is one command the tool offers, described in the language being +// rendered. +// +// The name comes from the program and the summary from the language file, the +// same split as a preset and its question. It exists because the page listing +// the commands was a hand copy of what tfg --help prints: on 2026-09-09 the +// program had ten commands and the page had nine, and nothing could tell, +// because Facts carried no list to compare against. +// +// Pad is the spaces between the name and the summary, worked out from the +// longest name rather than written into the template. The block is inside a +//
, so the alignment is content: a number in the template would be a
+// fourth copy of "the longest command is ten characters" and would go stale
+// the day an eleventh arrives.
+type Command struct {
+	Name    string
+	Pad     string
+	Summary string
+}
+
 // Download says which architectures a system actually gets.
 //
 // Both lists are here because they differ, and a page that flattened them into
@@ -101,6 +121,12 @@ type Facts struct {
 	Presets   []string
 	Downloads []Download
 
+	// Commands is what tfg --help prints, in the order it prints it, read out
+	// of that help rather than out of a list beside it. Taking it from the
+	// help is the point: it is the text a visitor is comparing the page
+	// against, so agreeing with anything else would prove the wrong thing.
+	Commands []string
+
 	// Year is fixed rather than taken from the clock. A footer that rendered
 	// the current year would make the committed pages differ from freshly
 	// rendered ones every first of January, and the guard would go red for a
@@ -172,19 +198,22 @@ type Page struct {
 // Dir is the path prefix. It is empty for the language served at the root,
 // which is the one search engines are pointed at by x-default.
 //
-// Endings and Terms are the two places where a word has to exist for every
-// value the program can produce. Endings is keyed by the exit code written out
-// in decimal, Terms by the kind or unit exactly as the registry spells it.
+// Endings, Terms, Presets and Commands are the places where a word has to
+// exist for every value the program can produce, and a missing one is an error
+// rather than a gap left in English. Endings is keyed by the exit code written
+// out in decimal, Terms by the kind or unit exactly as the registry spells it,
+// Presets by the identifier, and Commands by the name tfg --help prints.
 type Language struct {
-	Code    string            `json:"code"`
-	Name    string            `json:"name"`
-	Dir     string            `json:"dir"`
-	Words   map[string]string `json:"words"`
-	Endings map[string]string `json:"endings"`
-	Terms   map[string]string `json:"terms"`
-	Presets map[string]string `json:"presets"`
-	Pages   []Page            `json:"pages"`
-	Faq     []QA              `json:"faq"`
+	Code     string            `json:"code"`
+	Name     string            `json:"name"`
+	Dir      string            `json:"dir"`
+	Words    map[string]string `json:"words"`
+	Endings  map[string]string `json:"endings"`
+	Terms    map[string]string `json:"terms"`
+	Presets  map[string]string `json:"presets"`
+	Commands map[string]string `json:"commands"`
+	Pages    []Page            `json:"pages"`
+	Faq      []QA              `json:"faq"`
 }
 
 // Site is everything needed to render.
diff --git a/internal/site/view.go b/internal/site/view.go
index 8f07dfc..0b0215f 100644
--- a/internal/site/view.go
+++ b/internal/site/view.go
@@ -69,6 +69,7 @@ func (l Language) expand(f Facts) (Language, error) {
 	out.Endings = everyValue(l.Endings)
 	out.Terms = everyValue(l.Terms)
 	out.Presets = everyValue(l.Presets)
+	out.Commands = everyValue(l.Commands)
 
 	out.Pages = make([]Page, len(l.Pages))
 	for i, p := range l.Pages {
@@ -150,6 +151,35 @@ func (v view) PresetList() ([]Preset, error) {
 	return out, nil
 }
 
+// CommandList is every command tfg --help prints, in that order, summarised in
+// the language being rendered.
+//
+// The padding is worked out here rather than in the template. The block is
+// inside a 
 where spaces are content, and the width follows the longest
+// name - so an eleventh command longer than the tenth widens the column by
+// itself instead of leaving a template to be remembered.
+func (v view) CommandList() ([]Command, error) {
+	widest := 0
+	for _, name := range v.Facts.Commands {
+		if len(name) > widest {
+			widest = len(name)
+		}
+	}
+	out := make([]Command, 0, len(v.Facts.Commands))
+	for _, name := range v.Facts.Commands {
+		summary, ok := v.Lang.Commands[name]
+		if !ok {
+			return nil, fmt.Errorf("the command %q has no summary written in %s", name, v.Lang.Code)
+		}
+		out = append(out, Command{
+			Name:    name,
+			Pad:     strings.Repeat(" ", widest-len(name)+2),
+			Summary: summary,
+		})
+	}
+	return out, nil
+}
+
 // AllowedOf says what one setting accepts, in the language being rendered.
 //
 // The numbers come from the registry and the words from the language file. A
diff --git a/web/content/en/docs.html b/web/content/en/docs.html
index fb03ad6..ee60a4c 100644
--- a/web/content/en/docs.html
+++ b/web/content/en/docs.html
@@ -8,15 +8,7 @@ 

Documentation

What commands are there?

Each one does a single thing:

-
tfg generate    produce files, from a recipe or from flags
-tfg validate    check a recipe and write nothing
-tfg verify      check a directory against a manifest
-tfg cleanup     remove the files a manifest lists
-tfg recipe fmt  print a recipe in its settled shape
-tfg preset      build a set of files from a named test question
-tfg formats     list the formats this build supports
-tfg version     print the tool version
-tfg license     print the licence and what it means for generated files
+ {{ template "commandList" . }}
@@ -41,6 +33,7 @@

How do I generate a single file of an exact size?

--out <dir>directory to write into --seed <n>run seed. The same seed gives the same bytes --set <k>=<v>a format setting, repeatable + --damage <name>break the files on purpose, repeatable and applied in order. Run tfg damage for the list --expected <outcome>accept, reject, sanitize or unspecified --dry-runcount and show, write nothing at all --jsonwrite the manifest to standard output @@ -49,6 +42,50 @@

How do I generate a single file of an exact size?

+
+

How do I make a file that is broken on purpose?

+

+ Every other file this tool writes is correct by construction, which answers two of the three + questions an upload validator asks. --damage answers the third one - does the file + open at all. The file is produced normally and then broken, so it still has the size you asked + for. +

+
tfg generate --format png --size 2mb --damage zero-head --out ./out
+tfg generate --format png --size 2mb --damage zero-head:bytes=16 --out ./out
+

+ Settings go after a colon. The flag repeats, and the order you write them in is the order they + are applied. tfg damage lists what this build can do and what each one takes. +

+

In a recipe the key is a list, of names or of settings:

+
targets:
+  - id: broken
+    format: png
+    count: 5
+    size: 2mb
+    damage:
+      - zero-head
+      - type: zero-head
+        bytes: 16
+

+ A damaged file gets expected: reject in the manifest, with the damage recorded + beside it. Two things are refused before anything is written, because each would otherwise + put a file on disk that the manifest describes wrongly: +

+
    +
  • a file smaller than the damage needs, because it would come out unchanged
  • +
  • + expected: accept beside a damage, because nothing could meet it. Write + sanitize if the system under test is meant to repair the file, or + unspecified if that is the question you are asking +
  • +
+

+ A third one cannot be known in advance. If a damage runs and moves no byte, that file is + dropped rather than written - the run carries on, says which file it was, and ends with the + partial exit code. +

+
+

What does a recipe look like?

diff --git a/web/content/en/site.json b/web/content/en/site.json index 6b6ff3e..351aa25 100644 --- a/web/content/en/site.json +++ b/web/content/en/site.json @@ -97,6 +97,18 @@ "presets": { "size-boundaries": "Is a size limit enforced exactly where it is declared?" }, + "commands": { + "generate": "produce files, from a recipe or from flags", + "validate": "check a recipe and write nothing", + "verify": "check a directory against a manifest", + "cleanup": "remove the files a manifest lists", + "recipe fmt": "print a recipe in its settled shape", + "preset": "build a set of files from a named test question", + "formats": "list the formats this build supports", + "damage": "list the ways this build can break a file on purpose", + "version": "print the tool version", + "license": "print the licence and what it means for generated files" + }, "terms": { "oracleNone": "not applicable", "int": "any whole number", diff --git a/web/content/pl/docs.html b/web/content/pl/docs.html index c114957..27363ef 100644 --- a/web/content/pl/docs.html +++ b/web/content/pl/docs.html @@ -8,15 +8,7 @@

Dokumentacja

Jakie s膮 komendy?

Ka偶da robi jedn膮 rzecz:

-
tfg generate    tworzy pliki, z przepisu albo z flag
-tfg validate    sprawdza przepis i nic nie zapisuje
-tfg verify      sprawdza katalog wzgl臋dem manifestu
-tfg cleanup     usuwa pliki wypisane w manife艣cie
-tfg recipe fmt  wypisuje przepis w postaci uporz膮dkowanej
-tfg preset      buduje zestaw plik贸w z nazwanego pytania testowego
-tfg formats     wypisuje formaty, kt贸re ta wersja obs艂uguje
-tfg version     wypisuje wersj臋 narz臋dzia
-tfg license     wypisuje licencj臋 i to, co znaczy dla wygenerowanych plik贸w
+ {{ template "commandList" . }}
@@ -41,6 +33,7 @@

Jak wygenerowa膰 jeden plik o dok艂adnym rozmiarze?

--out <katalog>katalog, do kt贸rego trafiaj膮 pliki --seed <n>ziarno przebiegu. To samo ziarno daje te same bajty --set <k>=<v>ustawienie formatu, mo偶na powtarza膰 + --damage <nazwa>celowo psuje pliki, mo偶na powtarza膰, stosowane po kolei. List臋 wypisuje tfg damage --expected <wynik>accept, reject, sanitize albo unspecified --dry-runpolicz i poka偶, nie zapisuj niczego --jsonwypisz manifest na standardowe wyj艣cie @@ -49,6 +42,51 @@

Jak wygenerowa膰 jeden plik o dok艂adnym rozmiarze?

+
+

Jak zrobi膰 plik celowo zepsuty?

+

+ Ka偶dy inny plik, kt贸ry to narz臋dzie zapisuje, jest poprawny z definicji, co odpowiada na dwa + z trzech pyta艅 walidatora uploadu. --damage odpowiada na trzecie - czy plik w og贸le + si臋 otwiera. Plik powstaje normalnie i dopiero potem zostaje zepsuty, wi臋c dalej ma zam贸wiony + rozmiar. +

+
tfg generate --format png --size 2mb --damage zero-head --out ./out
+tfg generate --format png --size 2mb --damage zero-head:bytes=16 --out ./out
+

+ Ustawienia id膮 po dwukropku. Flag臋 mo偶na powtarza膰, a kolejno艣膰 zapisu jest kolejno艣ci膮 + stosowania. tfg damage wypisuje, co ta wersja umie i co ka偶de uszkodzenie + przyjmuje. +

+

W przepisie klucz jest list膮, nazw albo ustawie艅:

+
targets:
+  - id: broken
+    format: png
+    count: 5
+    size: 2mb
+    damage:
+      - zero-head
+      - type: zero-head
+        bytes: 16
+

+ Uszkodzony plik dostaje w manife艣cie expected: reject, a obok niego zapisane + uszkodzenie. Dwie rzeczy s膮 odmawiane, zanim cokolwiek powstanie, bo ka偶da zostawi艂aby na + dysku plik, kt贸ry manifest opisuje nieprawdziwie: +

+
    +
  • plik mniejszy ni偶 potrzebuje uszkodzenie, bo wyszed艂by nietkni臋ty
  • +
  • + expected: accept obok uszkodzenia, bo nic nie mog艂oby tego spe艂ni膰. Napisz + sanitize, je艣li system pod testem ma plik naprawi膰, albo + unspecified, je艣li w艂a艣nie o to pytasz +
  • +
+

+ Trzeciej rzeczy nie da si臋 wiedzie膰 z g贸ry. Je艣li uszkodzenie przebiegnie i nie ruszy ani + jednego bajtu, taki plik zostaje odrzucony zamiast zapisany - przebieg idzie dalej, m贸wi, + kt贸rego pliku to dotyczy艂o, i ko艅czy si臋 kodem cz臋艣ciowego wyniku. +

+
+

Jak wygl膮da przepis?

diff --git a/web/content/pl/site.json b/web/content/pl/site.json index 23aeefa..8328b64 100644 --- a/web/content/pl/site.json +++ b/web/content/pl/site.json @@ -97,6 +97,18 @@ "presets": { "size-boundaries": "Czy limit rozmiaru dzia艂a dok艂adnie tam, gdzie jest zadeklarowany?" }, + "commands": { + "generate": "tworzy pliki, z przepisu albo z flag", + "validate": "sprawdza przepis i nic nie zapisuje", + "verify": "sprawdza katalog wzgl臋dem manifestu", + "cleanup": "usuwa pliki wypisane w manife艣cie", + "recipe fmt": "wypisuje przepis w postaci uporz膮dkowanej", + "preset": "buduje zestaw plik贸w z nazwanego pytania testowego", + "formats": "wypisuje formaty, kt贸re ta wersja obs艂uguje", + "damage": "wypisuje sposoby, kt贸rymi ta wersja umie celowo zepsu膰 plik", + "version": "wypisuje wersj臋 narz臋dzia", + "license": "wypisuje licencj臋 i to, co znaczy dla wygenerowanych plik贸w" + }, "terms": { "oracleNone": "nie dotyczy", "int": "dowolna liczba ca艂kowita", diff --git a/web/public/docs/index.html b/web/public/docs/index.html index 4a36072..b54fd41 100644 --- a/web/public/docs/index.html +++ b/web/public/docs/index.html @@ -91,6 +91,7 @@

What commands are there?

tfg recipe fmt print a recipe in its settled shape tfg preset build a set of files from a named test question tfg formats list the formats this build supports +tfg damage list the ways this build can break a file on purpose tfg version print the tool version tfg license print the licence and what it means for generated files
@@ -117,6 +118,7 @@

How do I generate a single file of an exact size?

--out <dir>directory to write into --seed <n>run seed. The same seed gives the same bytes --set <k>=<v>a format setting, repeatable + --damage <name>break the files on purpose, repeatable and applied in order. Run tfg damage for the list --expected <outcome>accept, reject, sanitize or unspecified --dry-runcount and show, write nothing at all --jsonwrite the manifest to standard output @@ -125,6 +127,50 @@

How do I generate a single file of an exact size?

+
+

How do I make a file that is broken on purpose?

+

+ Every other file this tool writes is correct by construction, which answers two of the three + questions an upload validator asks. --damage answers the third one - does the file + open at all. The file is produced normally and then broken, so it still has the size you asked + for. +

+
tfg generate --format png --size 2mb --damage zero-head --out ./out
+tfg generate --format png --size 2mb --damage zero-head:bytes=16 --out ./out
+

+ Settings go after a colon. The flag repeats, and the order you write them in is the order they + are applied. tfg damage lists what this build can do and what each one takes. +

+

In a recipe the key is a list, of names or of settings:

+
targets:
+  - id: broken
+    format: png
+    count: 5
+    size: 2mb
+    damage:
+      - zero-head
+      - type: zero-head
+        bytes: 16
+

+ A damaged file gets expected: reject in the manifest, with the damage recorded + beside it. Two things are refused before anything is written, because each would otherwise + put a file on disk that the manifest describes wrongly: +

+
    +
  • a file smaller than the damage needs, because it would come out unchanged
  • +
  • + expected: accept beside a damage, because nothing could meet it. Write + sanitize if the system under test is meant to repair the file, or + unspecified if that is the question you are asking +
  • +
+

+ A third one cannot be known in advance. If a damage runs and moves no byte, that file is + dropped rather than written - the run carries on, says which file it was, and ends with the + partial exit code. +

+
+

What does a recipe look like?

diff --git a/web/public/pl/dokumentacja/index.html b/web/public/pl/dokumentacja/index.html index 21de9ad..84446c2 100644 --- a/web/public/pl/dokumentacja/index.html +++ b/web/public/pl/dokumentacja/index.html @@ -91,6 +91,7 @@

Jakie s膮 komendy?

tfg recipe fmt wypisuje przepis w postaci uporz膮dkowanej tfg preset buduje zestaw plik贸w z nazwanego pytania testowego tfg formats wypisuje formaty, kt贸re ta wersja obs艂uguje +tfg damage wypisuje sposoby, kt贸rymi ta wersja umie celowo zepsu膰 plik tfg version wypisuje wersj臋 narz臋dzia tfg license wypisuje licencj臋 i to, co znaczy dla wygenerowanych plik贸w
@@ -117,6 +118,7 @@

Jak wygenerowa膰 jeden plik o dok艂adnym rozmiarze?

--out <katalog>katalog, do kt贸rego trafiaj膮 pliki --seed <n>ziarno przebiegu. To samo ziarno daje te same bajty --set <k>=<v>ustawienie formatu, mo偶na powtarza膰 + --damage <nazwa>celowo psuje pliki, mo偶na powtarza膰, stosowane po kolei. List臋 wypisuje tfg damage --expected <wynik>accept, reject, sanitize albo unspecified --dry-runpolicz i poka偶, nie zapisuj niczego --jsonwypisz manifest na standardowe wyj艣cie @@ -125,6 +127,51 @@

Jak wygenerowa膰 jeden plik o dok艂adnym rozmiarze?

+
+

Jak zrobi膰 plik celowo zepsuty?

+

+ Ka偶dy inny plik, kt贸ry to narz臋dzie zapisuje, jest poprawny z definicji, co odpowiada na dwa + z trzech pyta艅 walidatora uploadu. --damage odpowiada na trzecie - czy plik w og贸le + si臋 otwiera. Plik powstaje normalnie i dopiero potem zostaje zepsuty, wi臋c dalej ma zam贸wiony + rozmiar. +

+
tfg generate --format png --size 2mb --damage zero-head --out ./out
+tfg generate --format png --size 2mb --damage zero-head:bytes=16 --out ./out
+

+ Ustawienia id膮 po dwukropku. Flag臋 mo偶na powtarza膰, a kolejno艣膰 zapisu jest kolejno艣ci膮 + stosowania. tfg damage wypisuje, co ta wersja umie i co ka偶de uszkodzenie + przyjmuje. +

+

W przepisie klucz jest list膮, nazw albo ustawie艅:

+
targets:
+  - id: broken
+    format: png
+    count: 5
+    size: 2mb
+    damage:
+      - zero-head
+      - type: zero-head
+        bytes: 16
+

+ Uszkodzony plik dostaje w manife艣cie expected: reject, a obok niego zapisane + uszkodzenie. Dwie rzeczy s膮 odmawiane, zanim cokolwiek powstanie, bo ka偶da zostawi艂aby na + dysku plik, kt贸ry manifest opisuje nieprawdziwie: +

+
    +
  • plik mniejszy ni偶 potrzebuje uszkodzenie, bo wyszed艂by nietkni臋ty
  • +
  • + expected: accept obok uszkodzenia, bo nic nie mog艂oby tego spe艂ni膰. Napisz + sanitize, je艣li system pod testem ma plik naprawi膰, albo + unspecified, je艣li w艂a艣nie o to pytasz +
  • +
+

+ Trzeciej rzeczy nie da si臋 wiedzie膰 z g贸ry. Je艣li uszkodzenie przebiegnie i nie ruszy ani + jednego bajtu, taki plik zostaje odrzucony zamiast zapisany - przebieg idzie dalej, m贸wi, + kt贸rego pliku to dotyczy艂o, i ko艅czy si臋 kodem cz臋艣ciowego wyniku. +

+
+

Jak wygl膮da przepis?

diff --git a/web/templates/partials.html b/web/templates/partials.html index c85d558..6a504a2 100644 --- a/web/templates/partials.html +++ b/web/templates/partials.html @@ -195,6 +195,11 @@ {{- end -}} +{{- define "commandList" -}} +

{{ range $i, $c := .CommandList }}{{ if $i }}
+{{ end }}tfg {{ $c.Name }}{{ $c.Pad }}{{ $c.Summary }}{{ end }}
+{{- end -}} + {{- define "downloadCta" -}}