Skip to content

feat(testdata): real-world corpus + ES CIF letter-check fix - #4

Merged
euskadi31 merged 2 commits into
mainfrom
feature/add-more-tests
Jul 3, 2026
Merged

feat(testdata): real-world corpus + ES CIF letter-check fix#4
euskadi31 merged 2 commits into
mainfrom
feature/add-more-tests

Conversation

@euskadi31

Copy link
Copy Markdown
Contributor

Summary

  • Add a hand-collected corpus of ~1300 real business identifiers (SIREN, SIRET, VAT, national register numbers for BE/DE/ES/FR/IT/NL) under testdata/, consumed by four new real-world tests (one per provider) plus a native-algorithm test under providers/euid/.
  • Package a derive-corpus CLI (regenerates the derived euid/*, siren/fr, registers/fr files from source lists) as an isolated module under a new Go workspace (go.work + hack/derive-corpus/go.mod).
  • Fix a real defect surfaced by the corpus: the ES CIF "either" group (C,D,F,G,J,L,M,N,U,V) now accepts both digit- and letter-check forms. Foreign entities like ArcelorMittal Spain N0181056C previously failed.

Testdata layout

testdata/
  registers/<cc>-valide.txt   raw national register (source of truth per country)
  euid/<cc>-valide.txt        derived, "<CC><REGISTER>." prefix
  vat/<cc>-valide.txt         VAT numbers (~25 per country, hand-collected)
  siret/fr-valide.txt         FR SIRETs (source of truth for FR)
  siren/fr-valide.txt         derived from SIRETs (siret[:9], dedup)
  MANIFEST.md                 sources, counts, gaps

Countries covered (initial pass): BE (42), DE (100), ES (26), FR (113 SIRETs → 113 SIRENs), IT (100), NL (81) for registers; ~25 VAT each. Remaining 21 EU countries: to be added in a later PR.

Data-quality process

Per the request: collect numbers from reliable sources first, run the tests second, triage failures one by one. That surfaced two things:

  1. NL Wikipedia infobox VAT values were fabricated — 23 of 25 initial numbers failed the mod-11 checksum. Replaced with btw-zoeken.nl-verified values (18 remaining). No lib bug.
  2. ES CIF N prefix (foreign entities) rejected legitimate letter-check CIFs — the "either" group in cifCheck/esCIF was hard-coded to digit-only. Fixed to accept both forms, matching the Spanish tax authority docs. Regression tests added in vat_test.go and euid_test.go.

Workspace / tooling

  • go.work + hack/derive-corpus/go.mod isolate the CLI so its future deps don't leak into the library module.
  • CI's go-version-file switched from go.mod to go.work.
  • Makefile's generate target now loops over both modules like build/test/tidy.
  • hack/derive-corpus fully lint-clean (wrapcheck-wrapped errors, 0o600 file perms, log-based CLI output).

Test plan

  • make test — 2027 tests, 100 % statement coverage across every provider package.
  • make lint — 0 issues.
  • go run ./hack/derive-corpus idempotent (no diff on re-run).
  • go run ./hack/derive-corpus works both from repo root and from inside hack/derive-corpus/.
  • Adversarially verified via a 3-lens workflow (workspace shape / build & run / Makefile & lint) — all three findings are addressed in this PR.

Commits

  1. fix(vat,euid): accept letter check for ES CIF "either" group — isolated lib bugfix + regression tests.
  2. feat(testdata): real-world business-id corpus + integration tests — corpus, tests, workspace, CLI.

euskadi31 added 2 commits July 3, 2026 14:46
The `C,D,F,G,J,L,M,N,U,V` letter group is documented by the Spanish tax
authority as accepting either a digit or letter check-digit. The
existing code rejected the letter form outright, so real-world CIFs
like ArcelorMittal Spain `N0181056C` (foreign entity, letter-check)
failed validation.

Fix: the default arm of `cifCheck` / `esCIF` now accepts either form.
ABEH stays digit-only and PQRSW stays letter-only, matching the AEAT
tables.

Regression tests added under both providers.
Add a hand-collected corpus of ~1300 real business identifiers (SIREN,
SIRET, VAT, national register numbers for BE/DE/ES/FR/IT/NL) sourced
from official registries, Wikipedia infoboxes and corporate legal
notices. The corpus lives under testdata/ and is consumed by four new
real-world tests (one per provider) plus a native-algorithm test under
providers/euid/. This gives the library a broader real-world safety
net than the current ~10 known-values-per-country coverage.

Layout:
  testdata/
    registers/<cc>-valide.txt   raw national register (source of truth)
    euid/<cc>-valide.txt        derived from registers, "<CC><REG>." prefix
    vat/<cc>-valide.txt         hand-collected VAT numbers (~25 per country)
    siret/fr-valide.txt         FR SIRETs (source of truth for FR)
    siren/fr-valide.txt         derived from SIRETs (siret[:9], dedupped)
    MANIFEST.md                 sources, counts, gaps

The derived files (euid/, siren/fr, registers/fr) are regenerated by
`go run ./hack/derive-corpus`, itself packaged as a separate module
under a Go workspace so its dev-tool dependencies stay isolated from
the library.

Countries covered (initial pass): BE (42), DE (100), ES (26), FR (113
SIRETs), IT (100), NL (81) for registers; ~25 VAT each. Remaining 21
EU countries: to be added later.

Data-quality notes:
- Numbers are collected first, tests run second, and failing entries
  are triaged one by one: real lib bug → fix; wrong transcription →
  remove. That process surfaced the ES CIF letter-check fix (separate
  commit) and revealed a batch of fabricated Wikipedia-sourced NL VAT
  numbers, replaced by btw-zoeken.nl-verified values.

Workspace / tooling:
- go.work + hack/derive-corpus/go.mod isolate the CLI.
- CI's go-version-file switched from go.mod to go.work.
- Makefile's `generate` target now loops over both modules like the
  other targets.

Tests: 2027 passing, 100 % statement coverage kept across all provider
packages.
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 28661634249

Coverage decreased (-5.8%) to 94.171%

Details

  • Coverage decreased (-5.8%) from the base build.
  • Patch coverage: 149 uncovered changes across 1 file (10 of 159 lines covered, 6.29%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
hack/derive-corpus/main.go 149 0 0.0%
Total (3 files) 159 10 6.29%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 2556
Covered Lines: 2407
Line Coverage: 94.17%
Coverage Strength: 55.78 hits per line

💛 - Coveralls

@euskadi31
euskadi31 merged commit c6b7a8f into main Jul 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants