Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 48 additions & 8 deletions .github/scripts/sign_release.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,22 @@ def powershell(script):
capture_output=True, text=True)
if out.returncode != 0:
raise SystemExit("sign_release: powershell failed:\n%s" % out.stderr.strip())
# PowerShell errors are NON TERMINATING by default, so a script can print
# a page of complaints and still exit zero. Reading stderr only on a
# non-zero code therefore threw away the one sentence that said what went
# wrong, and left the caller looking at empty output with no reason for it.
#
# It cost an hour on 2026-09-09 signing v0.3.0: the certificate lookup came
# back empty and the script blamed a missing card, while the card was in
# the reader and readable. The complaint was there the whole time and
# nothing printed it. O200.
#
# A note rather than a failure, because a warning is not a refusal and the
# caller may have asked something that legitimately produces one.
if out.stderr.strip():
print(" powershell also said:")
for line in out.stderr.strip().splitlines():
print(" %s" % line)
return out.stdout


Expand Down Expand Up @@ -170,17 +186,41 @@ def signing_thumbprint(pin):
the only selector it takes, and the repository pins SHA-256 because that is
the digest worth pinning. Resolving one to the other here means the two can
never drift apart in a configuration file.

THE STORE IS OPENED THROUGH .NET RATHER THAN THROUGH THE Cert: DRIVE, and
that is a measurement rather than a preference. The drive is provided by
Microsoft.PowerShell.Security, which Windows PowerShell 5.1 only loads when
PSModulePath points at its own module directory - and a 5.1 launched from
inside pwsh 7 is handed pwsh's PSModulePath instead. Measured 2026-09-09 on
this machine, from a python started under pwsh:

Get-ChildItem Cert:\\CurrentUser\\My -> 0, plus
"Cannot find drive. A drive with the name 'Cert' does not exist."
X509Store('My','CurrentUser') -> 11

Both ended with code ZERO, because a PowerShell error is non terminating -
so the script saw empty output and reported a missing card while the card
was in the reader. It cost an hour signing v0.3.0 and the release went out
through Git Bash as a workaround. X509Store is in the runtime rather than
in a module, so it does not depend on which shell started which. O200.
"""
script = (
"$out = @(); "
"Get-ChildItem Cert:\\CurrentUser\\My, Cert:\\LocalMachine\\My "
"-ErrorAction SilentlyContinue | Where-Object { "
" $_.Extensions.EnhancedKeyUsages.Value -contains '%s' } | ForEach-Object { "
" $h = [System.Security.Cryptography.SHA256]::Create().ComputeHash($_.RawData); "
" $out += [pscustomobject]@{ "
" sha256 = (($h | ForEach-Object { $_.ToString('x2') }) -join ''); "
" thumb = $_.Thumbprint; subject = $_.Subject; "
" notAfter = $_.NotAfter.ToString('s') } "
"foreach ($where in 'CurrentUser', 'LocalMachine') { "
" $store = New-Object System.Security.Cryptography.X509Certificates.X509Store('My', $where); "
" try { $store.Open('ReadOnly') } catch { continue }; "
" foreach ($c in $store.Certificates) { "
" $eku = @(); "
" foreach ($x in $c.Extensions) { "
" if ($x -is [System.Security.Cryptography.X509Certificates.X509EnhancedKeyUsageExtension]) { "
" foreach ($u in $x.EnhancedKeyUsages) { $eku += $u.Value } } }; "
" if ($eku -notcontains '%s') { continue }; "
" $h = [System.Security.Cryptography.SHA256]::Create().ComputeHash($c.RawData); "
" $out += [pscustomobject]@{ "
" sha256 = (($h | ForEach-Object { $_.ToString('x2') }) -join ''); "
" thumb = $c.Thumbprint; subject = $c.Subject; "
" notAfter = $c.NotAfter.ToString('s') } }; "
" $store.Close() "
"}; $out | ConvertTo-Json -Compress" % CODE_SIGNING_OID
)
entries = json.loads(powershell(script).strip() or "[]")
Expand Down
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,33 @@ because it turns other people's test suites red.

## [Unreleased]

### Fixed

- **Asking for damaged files and declaring they will be accepted is now refused
on the command line too.** A damaged file is one a reader was measured to
refuse, so `--expected accept` beside `--damage` asks for something nothing
can deliver.

A recipe saying the same thing has always been refused. The command line was
not: it wrote the files and recorded in the manifest that a deliberately
broken file should be accepted, which is the one place this tool must not say
something untrue. `tfg generate --damage zero-head --expected accept` now
ends with exit code `2` and writes nothing, and a recipe still ends with `3`
and names the target the problem is in.

Only `accept` is refused. `reject` is what damage already means, and
`sanitize` and `unspecified` are both real questions to ask about a broken
file - a system under test may be meant to repair it, or that may be the
point of the test - so all three still work, as does `--expected accept` on
files that are not damaged.

- **The documentation website lists every command the tool has.** `tfg damage`
arrived in 0.3.0 and the page describing the commands still showed the other
nine, because that list was written out by hand. The page now takes the list
from the program itself, so a command added later cannot go missing from it,
and the site has a section explaining how to produce a file that is broken on
purpose.

## [0.3.0] - 2026-09-09

### Breaking
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,7 @@ tfg generate --format txt --size 1mb settings come from the flags
| `--out <dir>` | directory to write into. Default `.` |
| `--seed <n>` | run seed. The same seed gives the same bytes |
| `--set <k>=<v>` | a format setting, repeatable: `--set width=1920 --set height=1080` |
| `--damage <name>` | break the files on purpose, repeatable and applied in order. Run `tfg damage` for the list |
| `--expected <outcome>` | `accept`, `reject`, `sanitize` or `unspecified` |
| `--expected-reason <r>` | why that outcome, from the closed list below |
| `--preset <id>` | build the set a named test question calls for |
Expand Down Expand Up @@ -350,6 +351,11 @@ The smallest file a damage can be given follows its settings, so the column is
measured with the defaults. Ask for less and the run is refused before anything is
written, naming a size that would work.

Asking for `--expected accept` beside a damage is refused too, because nothing
could meet it. Write `sanitize` if the system under test is meant to repair the
file, or `unspecified` if that is the question you are asking - both of those,
and `reject`, work as they always did.

## 📜 Recipes

A recipe is a YAML file describing a whole run. Commit it beside your tests and
Expand Down
13 changes: 13 additions & 0 deletions internal/cli/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,19 @@ func classifyRequest(err error) (int, bool) {
if errors.As(err, &tooSmall) {
return ExitFormat, true
}
// Damaging a file and declaring it will be accepted is two flags that
// cancel each other, which is a fault in the invocation rather than a
// request no format can meet - the same conflict stands for all of them.
// A script reading ExitFormat goes looking for another format or another
// size, and neither of those is the fix. Owner's call on 2026-09-09.
//
// Anything arriving here came off the command line: a recipe declaring the
// same pair is refused while the recipe is read, with the address of the
// target and code 3 beside its other problems. O199.
var impossible *damage.ExpectationConflictError
if errors.As(err, &impossible) {
return ExitUsage, true
}
if code, ok := classifyFormat(err); ok {
return code, true
}
Expand Down
46 changes: 46 additions & 0 deletions internal/damage/refusals.go
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,52 @@ func (e *NoChangeError) Error() string {
return e.What() + ". " + e.Why()
}

// RuledOutExpectation is the one declared outcome a damaged file cannot have.
//
// It is spelled here rather than imported because internal/manifest sits
// beside this package rather than under it, so the two cannot see each other.
// TestTheOutcomeDamageRulesOutIsTheOneTheManifestKnows compares this against
// manifest.OutcomeAccept and against the list a recipe accepts, which is what
// stops three spellings of one word from drifting apart.
const RuledOutExpectation = "accept"

// ConflictsWithExpectation is the refusal a target earns by damaging its files
// and declaring they will be accepted, or nil when there is no conflict.
//
// It lives here, on the chain, because it is a fact about damage rather than
// about either surface - and both surfaces ask it. Measured on 2026-09-09 with
// the check living in the recipe reader alone: a recipe was refused with code
// 3 while the identical run off the command line ended with code 0 and wrote a
// manifest saying a deliberately broken file should be accepted. The recipe
// reader still asks first, so it keeps reporting this beside every other
// problem of that recipe and with the address of the target - what changed is
// that the engine asks too, so no surface can get past it. See O199.
// Two shapes of one rule, and the pair is deliberate. The engine wants an
// error to hand upwards, while the recipe reader wants the parts - What, Why
// and Instead - to lay out beside the other problems of that recipe.
//
// Written as two functions rather than one returning the concrete type,
// because that one would be the typed nil trap: a nil *ExpectationConflictError
// placed in an error interface is NOT a nil error. Measured 2026-09-09 on a
// four case program - "reject", "sanitize", "" and "accept" all came back
// err != nil - so the engine would have refused EVERY target, damaged or not,
// and the guard beside this one asserts exactly that it does not.
func (c Chain) ConflictsWithExpectation(expected string) error {
if bad := c.ExpectationConflict(expected); bad != nil {
return bad
}
return nil
}

// ExpectationConflict is the same question answered with the refusal itself,
// or nil. For a caller that needs the parts rather than an error.
func (c Chain) ExpectationConflict(expected string) *ExpectationConflictError {
if len(c) == 0 || expected != RuledOutExpectation {
return nil
}
return &ExpectationConflictError{Outcome: expected}
}

// ExpectationConflictError is a target that damages a file and expects it to
// be accepted.
//
Expand Down
36 changes: 36 additions & 0 deletions internal/engine/damage.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,24 @@ import (
"github.com/donislawdev/TestingFilesGenerator/internal/manifest"
)

// checkDamage is every refusal a damaged target can earn during planning.
//
// One entry rather than two calls side by side in engine.go, and that is the
// same measurement this file was cut out for: adding the second call put
// engine.go at 411 lines of code against a ceiling of 408, and the answer to a
// ceiling is a cut rather than a larger number. These two belong together
// anyway - both are "what makes this target impossible before a byte is
// written", which is one subject.
//
// The floor first, because it names a number a person can act on. A target
// earning both refusals gets that one.
func checkDamage(t *Target) error {
if err := checkDamageFloor(t); err != nil {
return err
}
return checkDamageExpectation(t)
}

// checkDamageFloor refuses a file smaller than the damage it was given.
//
// Here rather than at the moment of writing, and that is the point: a file
Expand Down Expand Up @@ -46,6 +64,24 @@ func checkDamageFloor(t *Target) error {
return nil
}

// checkDamageExpectation refuses a target that breaks its files and declares
// they will be accepted.
//
// Here as well as in the recipe reader, and that is the whole point of it. The
// condition is one function on the chain, so this is a second CALLER rather
// than a second copy - what it buys is that the command line reaches it, and
// the command line never reads a recipe. Measured on 2026-09-09 before this
// existed: the recipe was refused with code 3 while
// --damage zero-head --expected accept ended with code 0 and wrote a manifest
// claiming a deliberately broken file should be accepted. O199.
//
// The window cannot reach this today - damage sits on the generate screen and
// the expectation on the recipe screen - and being here rather than in the
// reader is what covers it on the day those two meet.
func checkDamageExpectation(t *Target) error {
return t.Damage.ConflictsWithExpectation(t.Expected)
}

// damageFor records what was broken about this file, with the settings
// resolved rather than as written.
//
Expand Down
2 changes: 1 addition & 1 deletion internal/engine/engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ func settleTarget(t *Target, opt Options, seen map[string]bool) (format.Descript
// After the draw, because a range arrives here carrying only a count and a
// damage floor has to be judged against the sizes that will really be
// written.
if err := checkDamageFloor(t); err != nil {
if err := checkDamage(t); err != nil {
return format.Descriptor{}, err
}
return desc, nil
Expand Down
89 changes: 89 additions & 0 deletions internal/guard/damagerefused_test.go
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
package guard

import (
"bytes"
"context"
"errors"
"os"
"path/filepath"
"slices"
"strings"
"testing"

"github.com/donislawdev/TestingFilesGenerator/internal/cli"
"github.com/donislawdev/TestingFilesGenerator/internal/damage"
"github.com/donislawdev/TestingFilesGenerator/internal/engine"
"github.com/donislawdev/TestingFilesGenerator/internal/format"
Expand Down Expand Up @@ -313,6 +317,91 @@ targets:
}
}

// The command line refuses that pair as well, and writes nothing.
//
// The guard above asks recipe.Parse and only recipe.Parse, and that was enough
// to be green through a build where this was broken. Measured on 2026-09-09 on
// the 0.3.0 binary: the recipe was refused with code 3 while
// --damage zero-head --expected accept ended with code 0 and left a manifest
// on disk saying a deliberately damaged file should be accepted - the tool
// lying in the one place its value lives. O199.
//
// Both halves are needed. Asking only the refusal would pass for a build that
// refuses damage beside any expectation at all, and asking only that files are
// written would pass for one that refuses nothing - so the second loop is the
// wall detector and the first is the hole detector.
//
// The count of files is asked rather than the exit code alone: a refusal that
// arrives after the writing has started is a refusal that came too late, and
// the code by itself cannot tell those apart.
func TestTheCommandLineRefusesDamageBesideAcceptToo(t *testing.T) {
run := func(t *testing.T, extra ...string) (int, string, int) {
t.Helper()
dir := t.TempDir()
var out, errOut bytes.Buffer
args := append([]string{
"generate", "--format", "txt", "--size", "100",
"--damage", damage.ZeroHead, "--out", dir,
}, extra...)
code := cli.Run(context.Background(), args, &out, &errOut)
written, err := os.ReadDir(dir)
if err != nil {
t.Fatalf("reading the output directory: %v", err)
}
return code, errOut.String(), len(written)
}

code, said, files := run(t, "--expected", damage.RuledOutExpectation)
if code != cli.ExitUsage {
t.Errorf("damage beside %q ended with %d, expected %d - two flags that cancel each other are a fault in the invocation\nstderr: %s",
damage.RuledOutExpectation, code, cli.ExitUsage, said)
}
if files != 0 {
t.Errorf("the run was refused and still left %d file(s) behind", files)
}
if !strings.Contains(said, damage.RuledOutExpectation) {
t.Errorf("the refusal does not name the word it turned down: %s", said)
}

// The other three are legitimate questions about a broken file, and a
// build refusing them would be a wall rather than this rule.
for _, outcome := range []string{"reject", "sanitize", "unspecified"} {
if code, said, _ := run(t, "--expected", outcome); code != cli.ExitOK {
t.Errorf("--expected %s beside damage ended with %d rather than %d: %s",
outcome, code, cli.ExitOK, said)
}
}
// And the expectation on its own is untouched. Every case above carries a
// damage, so a build that refused accept for every run whatsoever would
// look correct from all of them.
dir := t.TempDir()
var out, errOut bytes.Buffer
if code := cli.Run(context.Background(), []string{
"generate", "--format", "txt", "--size", "100",
"--expected", damage.RuledOutExpectation, "--out", dir,
}, &out, &errOut); code != cli.ExitOK {
t.Errorf("--expected %s with nothing damaged ended with %d rather than %d, which makes this a wall rather than a rule about damage: %s",
damage.RuledOutExpectation, code, cli.ExitOK, errOut.String())
}
}

// The outcome damage rules out is the one the manifest and the recipe know.
//
// Three spellings of one word live in three packages that cannot import each
// other - damage sits beside manifest rather than under it - so this compares
// them rather than leaving them to drift. A rename in one place turns this red
// instead of quietly producing a build where nothing is ever refused.
func TestTheOutcomeDamageRulesOutIsTheOneTheManifestKnows(t *testing.T) {
if damage.RuledOutExpectation != manifest.OutcomeAccept {
t.Errorf("damage rules out %q and the manifest calls it %q, so nothing would ever match",
damage.RuledOutExpectation, manifest.OutcomeAccept)
}
if !slices.Contains(recipe.Outcomes(), damage.RuledOutExpectation) {
t.Errorf("damage rules out %q and a recipe does not accept that word at all: %v",
damage.RuledOutExpectation, recipe.Outcomes())
}
}

// A damage the build does not know is refused while the recipe is read, and
// the refusal names what there is.
//
Expand Down
Loading
Loading