diff --git a/docs/adr/0016-master-binding-authority.md b/docs/adr/0016-master-binding-authority.md new file mode 100644 index 000000000..38dcd5228 --- /dev/null +++ b/docs/adr/0016-master-binding-authority.md @@ -0,0 +1,450 @@ +# ADR 0016: Master binding is deterministic, identifier-first, and never auto-resolves + +## Status + +Accepted for the shared binding contract in `bridge-tally-core`, consumed by the +agent/MCP layer and by the desktop source-draft flow. Master creation, ledger or +stock-item write authority, voucher generation, posting, and any model-assisted +or scored matching remain rejected without separate evidence. + +## Context + +Four document-import engagements have run to completion — one failed, three +clean, 152 vouchers, zero rejections. **Not one failure was OCR, parsing, or +model quality. Every failure was binding the document's entities to the target +book's masters.** + +- One engagement rejected 61 vouchers: seven ledger masters were missing, four + of them near-misses of ledgers that already existed, and suspense was posted + to an account that did not exist. +- One bank engagement survived only because ambiguous truncated payees were + parked in suspense rather than guessed. +- One sales engagement met five missing stock items, three near-duplicate sales + ledgers differing by one character and word order, and a customer whose ledger + name differed from the source name entirely. **Fuzzy name matching offered + three candidates and all three were the wrong person.** Only a mobile number + the operator had embedded in the ledger name identified them, and it matched + exactly. + +Bridge has two surfaces that need this and is currently growing two answers: + +- `agent_import.rs::master_match` (MCP `validate_masters`) compares a + case-and-whitespace-folded key by equality or prefix, and — for a near-miss — + emits `exact_live_spelling: candidates.first()`. That field names one + candidate as *the* live spelling with no evidence that it is; it is a silent + auto-resolution of exactly the case that caused the 61-voucher failure. +- PR #276 gave the desktop the opposite and correct behaviour: an operator loads + the observed ledger list and explicitly assigns a target, with a fresh reread + proving the selection still exists. It deliberately ranks nothing — but it + also narrows nothing, so the operator faces the entire catalog per entry. + +Neither surface matches on an embedded identifier, which is the one rule that +would have decided the case that defeated fuzzy matching. + +## Decision + +Binding is a **pure, deterministic function in `bridge-tally-core`** over +(one company's observed masters of one class, the entities named by one source +document). It performs no I/O, holds no transport handle, calls no model, and +depends on nothing above `bridge-tally-primitives`. Both surfaces consume it; +neither reimplements it. + +### 1. Inputs are valid by construction + +`MasterCatalog::new(class, names)` and `SourceEntity::new(...)` parse at the +boundary and fail closed with a typed `MasterBindingError`. `bind()` then takes +already-valid inputs and returns a report with no `Result`, so no caller can +re-check or compensate differently (P3). + +The constructor refuses, rather than degrades, on: + +- **an empty catalog** — `CatalogEmpty`. Binding against a book that was never + read is the May failure exactly, and it is now a typed error rather than a + report full of "missing"; +- **a catalog carrying the same name twice** — `CatalogDuplicateName`. If a name + does not identify one master, nothing downstream is meaningful; +- **an identifier hint that yields no identifier** — `IdentifierHintUnusable`. A + hint that silently does nothing is a trap (P7); +- bounds violations on entry count, entity count, name length, **total catalog + name bytes**, **total source name bytes**, and **hint count**. The byte bounds + are not redundant with the count and length ones: 20,000 names of 16,384 + characters satisfies both and is 327 MB before the constructor builds its + keys, tokens and four indexes over them. The catalog's is accumulated as the + iterator is consumed, so a lazy catalog fails before the next name is retained + rather than after all of them are. The source side had no such bound at all + until a review asked why only one side of an equally untrusted pair carried + one — 40,000 entities of 16,384 characters is two and a half gigabytes of + names, each individually valid — and it is now checked at `bind`, the boundary + where the collection first becomes this module's problem, as + `SourceNamesTooLarge`. The last is checked as the hints arrive rather than on the finished + set: hints deduplicate, so a million repeated ones fold to a single identifier + and the finished set never exceeds its bound, while every one of them has + already been scanned and copied. Each hint yields at least one identifier or + is refused outright, so the eager bound rejects nothing the late one admitted. + +A softer collision — two masters differing only in case, whitespace runs, or +dash and quote style — does **not** fail the catalog. It is carried as a +per-entity ambiguity instead, so every other master still binds and the +colliding pair surfaces in the unbound list where an operator can see it. +Failing a whole read to report one collision would block all the work it was +performed for. + +`MasterClass` is `Ledger` or `StockItem`. Both classes failed in practice and +the identifier rules are identical for both, but **the name fold is not**: +§9.4d measured ledgers, and whether a stock item matches by the same rule was +never sent. So a folded stock-item name may *suggest* and may not resolve — +byte equality is unaffected, since it needs no fold. The class is carried both +so a report cannot be applied to the wrong catalog and because the evidence +behind the two differs. + +### 2. The identifier is the key; the name is a hint + +Operators bury phone numbers, account numbers, and part codes inside master +names. Where such an identifier is present it is matched **before** any name +comparison, because a name comparison on the same pair is actively misleading. + +Two identifier shapes are extracted, deterministically, from both source names +and master names, and additionally accepted from the caller when the source +document carries an identifier outside the name (a statement's payment +reference, say): + +- **Numeric** — a maximal digit run, allowing internal hyphens and slashes, of + at least `MIN_NUMERIC_IDENTIFIER_DIGITS` (8) digits. Canonical form is the + digits alone, so a punctuated account number and a plain one agree. Internal + *spaces* are deliberately not allowed: fusing separated digit groups would + manufacture identifiers out of unrelated numbers, so a spaced value fails + closed to a near-miss instead. Eight digits is the threshold at which a year, + a rate, a house number and a masked last-four cannot qualify — a last-four + written as digits falls through to near-miss rather than binding two accounts + that share four digits. +- **Code** — a token holding at least two letters and at least + `MIN_CODE_IDENTIFIER_DIGITS` (3) digits, of at least + `MIN_CODE_IDENTIFIER_CHARS` (8) alphanumeric characters, and not a period + label. Canonical form is uppercase alphanumerics, so a punctuated part number + and an unpunctuated one agree. + + These thresholds were raised twice under review, from 4/2/1. Enumerating the + period spellings that must not become identifiers — `FY25`, then `APR2025`, + then `SEPTEMBER2025` — kept losing to the next spelling, so length carries + what a list of prefixes could not: a registration code clears eight + alphanumerics with three digits, and a period label does not. **Measured + against 485 live ledger names, exactly one yields a code identifier at all**, + so the cost of the strictness is nothing observed. + +**A token carrying letters never yields a standalone numeric**, whether or not +it qualified as a code. Otherwise `Part A12345678` reaches an unrelated +`Bank 12345678` through the one-letter gap the code test rejects: a token +identifies by its whole shape or not at all. + +**A mask is a mask however it is spelled, and wherever it is written.** A value +carrying mask punctuation (`****`, `####`) or a run of one repeated letter +(`XXXX`) exposes a suffix rather than a number, and that suffix is no more +identifying a space away than joined: `XXXX 12345678` is the same statement as +`XXXX12345678`. A mask therefore suppresses the token that follows it, in both +spellings and for both identifier shapes. Two unrelated ledgers sharing a masked +last-eight must reach a near-miss, never a bind. + +A token carrying no alphanumeric content is a **delimiter**, and a delimiter +does not end a mask: `XXXX - 12345678` says what `XXXX 12345678` says. Reading +the mask state token by token let a single `-` or `/` clear it and walk the +suffix out as a whole account number. An ordinary word does end a mask, or +nothing downstream of one could identify anything again. + +**An identifier may only be built from characters the token actually has.** +Canonical form keeps ASCII alphanumerics, and the digit-run split keeps ASCII +digits, so anything else in a token is discarded in silence — and what survives +is an identifier the name never contained. A name in another script fused to +`AB12345678` yielded that code and reached an unrelated bank; `12345678` +followed by Devanagari numerals yielded that number and did the same. In both +cases the ASCII spelling of the same shape never would. + +So the admitted set is **positive**: a token yields an identifier only if it is +ASCII apart from the dash variants this module already folds as separators. +Guarding "non-ASCII letters" was the first attempt and was too narrow — +`char::is_alphabetic` is false for a Devanagari digit — which is the second time +in this module an ASCII-shaped class silently decided a non-ASCII question. The +question is not which scripts exist; it is which characters canonicalization is +entitled to drop. The books this binder reads carry Devanagari, Tamil and +Bengali ledger names, so the boundary is reached rather than theoretical. + +**Period labels are recognized by their numbers, not their words.** A token is +a period when every number in it reads as a year or a small ordinal — which +catches `SEPTEMBER2025` and `2025QUARTER1` that no cap on the alphabetic run +ever would, because a month name can be any length and a year cannot. A fiscal +range (`2025-2026`, `2025/2026`) is excluded before its digits are fused, since +stripping the separator produced an eight-digit run that no calendar reading +rejects. Written without any separator the range arrives as one run that the +splitting step never sees — `FY202425`, `FY20242025` — so a year followed by a +two- or four-digit year is read as a period in its own right. Otherwise a +missing `Purchases FY202425` identifier-binds to a sole live `Sales FY202425`. + +One narrow exclusion applies to the numeric shape: an eight-digit run that reads +as a calendar date in 1900–2199 is a date, not an identifier. Without it two +unrelated period-labelled masters fuse on their period. The exclusion can only +make a bind less likely, never more, which is the safe direction for a rule +whose failure mode is posting against the wrong party. + +**Coverage is a property of the client's naming habit, not of the problem.** +Measured across four catalogues: a retail motorcycle dealership carries an +embedded identifier in 91 of 214 ledgers (42%), because it literally names +customers that way; a B2B minerals trader, 0 of 105; a third catalogue, 0 of +470; and Bridge's own synthetic books, 0 of 470 until ten were seeded to give +the rule any live coverage at all. So this rule is a **first-pass check that is +decisive when it fires and absent more often than not** — it resolved a customer +three fuzzy name matches got wrong, and it can never be the primary key. The +binder must work with it absent, and does: name matching is not a fallback here +but the ordinary path. + +An identifier binds only when it is **unique on both sides**: exactly one +master in the catalog carries it, and the entity's identifiers select exactly +one master overall. Any conflict is `Ambiguous`, never a bind. This keeps the +rule safe in the case that motivates it — a shared identifier is evidence of a +naming collision the operator must see, not licence to choose. + +Identifier-first has one more guard. Where a decisive identifier points at one +master while the entity's name is byte-equal to a *different* master, two strong +signals disagree, and the disagreement is reported +(`IdentifierNameConflict`) rather than silently settled in the identifier's +favour. + +### 3. Name matching binds only on an exact or normalized-exact unique hit + +`Exact` is byte equality with the observed master name. `Normalized` is equality +under **Tally's own rule for when two master names are the same**, and only when +exactly one master shares it. + +**There are two folds, and which one may answer is the whole of this section.** + +The resolving fold implements exactly the equivalences +`TALLY_PROTOCOL_REFERENCE.md` §9.4d measured on **licensed TallyPrime 7.1** — +the SKU this writes to — by naming each spelling in a voucher and reading the +day book back to see which master it reached: + +- ASCII case folds; +- leading and trailing whitespace is ignored; +- an internal run of spaces collapses; +- **space, `-` and `/` are one separator**, in both directions. + +Everything else is exact on codepoints. The wide fold (`master_identity_key`) +carries more than that and may only offer candidates. + +**The two rules that matter are negative, and neither is guessable.** An **en +dash** and an **underscore** were sent and *rejected*: they are not separators +to Tally however much they look like ones, so a fold that treats "punctuation" +or "separators" as a class is wider than the gateway and merges masters it keeps +apart. And **canonical equivalence is not folded** — an NFD spelling of an NFC +master is a different master, consistent with the exact-codepoint finding +recorded against this release. + +**This section has been wrong twice, in both directions, and the record is +worth more than the conclusion.** + +It first claimed the fold "stops exactly where Tally stops" while resolving on +four transformations §9.4b marked UNVERIFIED — a Bridge guess wearing Tally's +authority. That was corrected by narrowing to the three §9.4b had measured, +which cost 420 of 995 mutation binds and withdrew `X - Y`, a common ledger +convention. + +Then the narrowing turned out to be over-strict, because §9.4b's scope is *Edit +Log 7.0 Educational* and this project writes to licensed 7.1. Measuring that SKU +directly (§9.4d) found the gateway wider: the reverse hyphen direction, leading +whitespace, collapsed runs and slash all match. The fold is symmetric again, one +key per side, and the asymmetric index the narrow version needed is gone. On the +mutation book **600 of 995** now bind, against 420 under the narrow fold. + +The lesson is not "measure more". It is that **the scope line of an inherited +measurement is part of the measurement**: §9.4b was accurate and its scope was +the thing being skipped, by me in one direction and then by the narrowing in the +other. + +**What still holds regardless of which way the evidence moves.** A fold that +merges two **live** masters never resolves — the pair is an ambiguity and both +surface (§4). Two masters differing only in case, trailing whitespace or +separator style collapse under the measured fold and are refused there, which is +also what Tally implies, since it would match that source name to either. + +**Trimming.** Neither side is trimmed on the way in: `validated_name` bounds a +name and returns it unchanged, and an observed master is retained byte for byte +because a caller writes it back. Whitespace is handled by the fold, not by +editing the stored text. + +This fold is deliberately **separate from the general comparison key**, which is +shared with other contracts for voucher numbers and voucher-type names. §9.4b +says nothing about those, and widening the shared fold to serve masters would be +the "never to make one caller's case pass" this ADR warns against. One fold per +notion of sameness, each named for the question it answers. + +**Nothing else binds.** There is no edit distance, no phonetic key, no token +stemming, and no similarity threshold anywhere in the implementation. + +### 4. Near-misses produce candidates and never resolve + +Every non-binding entity carries its candidates, each labelled with the **rule +that produced it** — `SharedIdentifier`, `NormalizedEqual`, `SourcePrefix`, +`CatalogPrefix`, or `SharedToken`. Candidates are ordered by rule and then by +name; **no candidate is marked best, first-choice, or `exact_live_spelling`, +and no numeric score is emitted at all.** + +A score is rejected as a matter of contract, not of tuning. A score invites a +threshold, a threshold auto-resolves, and auto-resolution is what put money +against the wrong parties. The vocabulary is therefore a *basis* — a fact about +which rule fired — and never a confidence value (P6: the marker is recorded, and +it records what was observed). + +`SharedToken` suppresses tokens that occur in more than +`COMMON_TOKEN_PERCENT` (10%) of a catalog of at least `COMMON_TOKEN_MIN_CATALOG` +(20) entries, so a catalog-wide word cannot pull in every master. The +suppression is measured from the catalog rather than from a built-in word list, +which keeps it free of language and domain assumptions. + +Candidates are capped at `MAX_CANDIDATES_PER_ENTITY` (25) with the true +`candidate_count` and an explicit `candidates_truncated` flag retained, so a +truncated list is never mistaken for a short one. + +### 4a. An empty candidate list is three different facts, and the producer says which + +`candidates` can be empty for three unrelated reasons, and they mean opposite +things to anyone deciding what to do next: + +| `reason` | what empty means | +| --- | --- | +| `NoCandidate` | no master resembles this name at all | +| `NoDiscriminatingCandidate` | `candidate_count` masters resemble it and none is separable — **many exist**, none is worth showing | +| any, with `candidates_truncated` | the list was cut, by the per-entity cap or by the report's aggregate byte budget | + +So `candidates.is_empty()` alone answers nothing. The disambiguators are +`reason`, `candidate_count` and `candidates_truncated`, and a consumer that +reads the empty vector as "nothing exists" is wrong in two cases out of three. + +This is stated here, in the producer's contract, rather than left to each +consumer to rediscover, because **it has already been got wrong twice by +different lanes**: the preparation screen rendered "0 possible ledgers are +listed first" over a family of 120, and the voucher-presence contract had to +add a paired test to stop its own rule collapsing into "no candidates means +unknown" — a reading that is right for the truncated case and wrong for +`NoCandidate`. + +It is the same defect class this ADR was written against: a refusal whose +neighbouring value reads as an answer. The vocabulary is deliberately explicit +so that "nothing survived to be shown" and "nothing exists" cannot be confused +by reading one field. + +**This is now a type as well as a doc.** `Candidates` is +`None | Listed | Truncated { found } | Withheld { found }`, so a consumer +matching it exhaustively is made to decide each case, and the wrong reading does +not compile rather than failing a test someone remembered to write. `listed()`, +`found()` and `is_incomplete()` cover the callers that do not need to match. +`is_incomplete()` is the predicate that matters: **true means the absence of a +listing is not the absence of a master**, and no consumer may report "nothing +like this is present" over it. + +It was taken before merge deliberately. The contract had not shipped, so this is +the cheapest the change would ever be; afterwards it would be a breaking change +to a published contract with three consumers behind it. The consumer who paid +for it measured its own cost at about thirty lines and reported that the change +made its code better rather than merely compatible — a hand-assembled +disjunction became an exhaustive match. + +**The fix stops at the crate boundary, and says so.** The MCP result carries an +explicit `listing` discriminator, because a model is precisely the caller that +would read an empty array as "no such ledger exists". The desktop DTO stays +flat: its screen already distinguishes the three cases and is tested on each, so +flattening there is a projection with a tested consumer rather than an +ambiguity. Neither boundary has the compiler behind it — this protects Rust +consumers, and the projections are the two places where that protection ends. + +### 5. Status vocabulary + +Per entity, exactly one of: + +| status | meaning | +| --- | --- | +| `Bound { catalog_name, basis }` | one master, decided by `Identifier`, `ExactName`, or `NormalizedName` | +| `Ambiguous { candidates, .. }` | more than one master is defensible, including every identifier conflict | +| `Unmatched { candidates, .. }` | no rule produced a candidate | + +`Ambiguous` and `Unmatched` are the **unbound list, which is the product**. Each +unbound entry carries the source name as given, a stable `safe_reason_code`, its +extracted identifiers as `unresolved_identity`, and its candidates. "Here is +what I could not bind, and why" is the operator's actual work item; it is not an +error path and is not logged as a failure. + +### 6. A fallback binding is constructed, never inferred + +An ambiguous entity must remain postable. `FallbackBinding::assign` accepts an +**unbound** entry and a catalog-verified fallback master (a suspense ledger), +and retains the entity's `unresolved_identity` so a later reallocation journal +can find it without re-reading the source. It cannot be constructed from a bound +entity, so "silently rebound something that already matched" is not a +representable state (P2). Binding itself never emits a fallback. + +### 7. Totals prove the run + +`BindingReport::totals()` reports `requested`, `bound`, `ambiguous`, +`unmatched`, and `unbound`. `requested == bound + unbound` and +`unbound == ambiguous + unmatched` are invariants asserted by test, matching the +control-total discipline that proved every clean engagement. + +### 8. Identity stays where identity is already proven + +The report names masters by their **observed name only**. It holds no GUID and +grants no authority. A caller that intends to act on a binding re-reads the +catalog and revalidates the selection through the existing admission path — +`StandardLedgerCatalog::bind_selected` plus a fresh +`StandardLedgerCatalogBinding::matches` — exactly as PR #276 already requires. +This preserves that PR's rule that matching text alone is never a selected or +approved target, and keeps GUIDs out of a portable crate that has no company +scope to check them against. + +Consequently a binding is a **proposal**, never an approval. It selects no +voucher, creates no master, and dispatches nothing. + +## Consequences + +- `agent_import.rs::master_match` and its private `master_key` are deleted and + `validate_masters` is re-expressed over the crate. `match_state` gains + `normalized` and `identifier` alongside `exact`, `near_miss` and `missing`, + and `exact_live_spelling` now appears only on a bound row. A caller reading + that field on a near-miss was reading a guess. +- The old implementation classified *every* normalized-equal name as a + near-miss, so a request differing from the live ledger only in case, + whitespace, or dash style produced a candidate list instead of an answer. + Those now bind and report the live spelling. +- `build_import_xml` and the approved-post recheck still admit **`exact` only**. + The import file carries the name verbatim, so a normalized or identifier bind + informs the operator without widening what may be written. This PR does not + move the write gate. +- The MCP result reports an unbound entity's `unresolved_identity` wrapped in + the same party-name marker as every other name, so egress redaction treats it + identically. It adds no exposure: those identifiers are extracted from the + requested name the same result already echoes. +- The desktop catalog load returns an advisory binding per source entry, so the + operator sees the few relevant ledgers rather than all of them. It confers no + authority: assignment still runs the unchanged apply path, which rereads the + catalog and proves the selection is current. An unusable capture narrows + nothing rather than failing a read the operator just performed. +- Stock items are covered by contract before a stock-item catalog read exists. + When that read lands it supplies names to the same constructor; nothing in + this contract changes. Until then the shipped consumers pass `Ledger`, so the + stock-item half of the recorded failure is designed for but not yet reachable + from a screen. +- Binding is pure computation over already-observed data, so P1's live-evidence + requirement is satisfied upstream by the catalog read that produces its input. + Its own tests are fabricated from a placeholder alphabet: they establish the + behaviour of the rules, and are not, and may not be presented as, evidence + about any Tally instance. + +## Alternatives rejected + +- **Fuzzy or scored matching (edit distance, trigram, phonetic).** Directly + disproven: on the case that mattered its three best candidates were three + different wrong people, and a fourth-ranked exact identifier was present. +- **Auto-resolving a single candidate.** A single candidate is exactly the + four-near-miss situation that rejected 61 vouchers. Uniqueness of a *guess* is + not evidence. +- **Building this in the MCP and migrating later.** The two surfaces would + diverge before the migration; the divergence has already begun in + `master_match` and this ADR ends it rather than duplicating it. +- **Putting binding in `bridge-tally-protocol`.** Binding parses no wire format + and needs no XML. `bridge-tally-core` is the portable contract layer and holds + the analogous reconciliation logic (P8: dependencies point inward). diff --git a/docs/agent/README.md b/docs/agent/README.md index abeb63dbf..4c0f6cf23 100644 --- a/docs/agent/README.md +++ b/docs/agent/README.md @@ -205,8 +205,20 @@ licence mode, or manually imported file, and only an unnumbered single-voucher 1. Call `voucher_schema` and produce a payload matching its schema. Transaction IDs are client-supplied, unique within the batch, and retained in the local import ledger. -2. Call `validate_masters` with every ledger name. Correct every `near_miss` - with the exact live spelling; Bridge never creates masters. +2. Call `validate_masters` with every ledger name. **`build_import_xml` admits + `exact` only**, so replace the payload name for every row that is not + `exact`, and never invent one: + - `normalized` or `identifier` — the row is bound. Copy its + `exact_live_spelling` into the payload verbatim; the live name may differ + from yours in case, spacing, dash or quote style, and the import file + carries whatever you send byte for byte. + - `near_miss` — the row is **not** bound and Bridge chose nothing. Pick from + `candidates`, each labelled with the rule that surfaced it. A single + candidate is still not a decision. Where `reason` is + `master_binding_no_discriminating_candidate`, the name reaches + `candidate_count` masters that it does not distinguish and none is listed; + use a more complete source name, or read the ledger list and choose. + - `missing` — no live ledger matched. Bridge never creates masters. 3. Call `build_import_xml` with the payload. It checks exact decimal balance, company date extent, live masters, and local journal integrity, repeats the full catalogue to reject intervening changes, then writes `/imports/.xml` and records an append-only diff --git a/docs/tally/TALLY_PROTOCOL_REFERENCE.md b/docs/tally/TALLY_PROTOCOL_REFERENCE.md index cc1e12a6f..56fed873f 100644 --- a/docs/tally/TALLY_PROTOCOL_REFERENCE.md +++ b/docs/tally/TALLY_PROTOCOL_REFERENCE.md @@ -1154,14 +1154,19 @@ symmetry is exactly the property the separator result does not have. | --- | --- | | ASCII case folding | **VERIFIED** — lowercase matched | | supplying a **space** where the master has a **hyphen** | **VERIFIED** — `BRIDGE PROBE LEDGER A` matched `BRIDGE-PROBE-LEDGER-A` | -| supplying a **hyphen** where the master has a **space** | **UNVERIFIED** — the reverse direction was never sent | +| supplying a **hyphen** where the master has a **space** | **UNVERIFIED here** — the reverse direction was never sent on this SKU. Measured **matched** on licensed 7.1, §9.4d | | one trailing space ignored | **VERIFIED** | | **two or more** trailing spaces ignored | **UNVERIFIED** — only one was sent | -| *leading* whitespace ignored | **UNVERIFIED** | -| runs of internal whitespace collapsed to one | **UNVERIFIED** — only a single space was tested | +| *leading* whitespace ignored | **UNVERIFIED here**. Measured **matched** on licensed 7.1, §9.4d | +| runs of internal whitespace collapsed to one | **UNVERIFIED here** — only a single space was tested. Measured **matched** on licensed 7.1, §9.4d | | non-ASCII case folding (Devanagari, Tamil, Bengali, Turkish dotted I) | **UNVERIFIED** | | **Unicode canonical equivalence (NFC/NFD)** | **MEASURED — folding it is wrong.** See below. | -| any other separator (underscore, en dash, `/`) treated as a space | **UNVERIFIED** | +| any other separator (underscore, en dash, `/`) treated as a space | **UNVERIFIED here**, and §9.4d splits it on licensed 7.1: `/` **matched**, underscore and en dash **rejected**. Not one row — do not fold them together | + +**A wider result exists for a different SKU.** §9.4d re-ran this measurement on **licensed +TallyPrime 7.1** and found the gateway folds more than these rows establish. It is a separate +section on purpose: these rows are about Edit Log 7.0 Educational, and absorbing a licensed-Silver +result into them would silently widen the scope of a measurement nobody repeated here. **The NFC/NFD row is the only one with evidence pointing the wrong way**, rather than no evidence at all, and it is the one most likely to be folded in by accident. @@ -1260,6 +1265,130 @@ Whether stock items, groups and voucher types match by the same rule is UNVERIFI says nothing about voucher numbers — a fold shared between master names and voucher numbers is assuming something nobody has measured. +### 9.4d Master-name matching on **licensed** TallyPrime 7.1 + +**VERIFIED 2026-09-12**, and it widens §9.4b rather than confirming it. §9.4b is inherited from a +2026-07-30 measurement on **Edit Log 7.0 Educational** and marks licensed TallyPrime UNVERIFIED. +This is that measurement re-run on the SKU this project actually writes to: **TallyPrime 7.1, +licence tier silver, `education_mode=false`**, ledgers, one lab company. + +**Method is §9.4b's own.** Import a voucher naming a folded spelling of a ledger that exists, and +let Tally answer: a created voucher means the name resolved, a `LINEERROR` naming that ledger +means it did not. Twelve variants in the first run and six more in the second described below, one +voucher each, then the **day book was read back** to record which master each voucher actually +posted against — the counters alone would not have said. Every created voucher was then deleted by +`REMOTEID` and the day read back empty (eight from the first run, two from the second). + +| Supplied against a live master | Licensed 7.1 | §9.4b on Educational | +| --- | --- | --- | +| exact | **matched** | matched | +| ASCII lowercase | **matched** | matched | +| one trailing space | **matched** | matched | +| a **space** where the master has a **hyphen** | **matched** | matched | +| a **hyphen** where the master has a **space** | **matched** | *UNVERIFIED* | +| leading whitespace | **matched** | *UNVERIFIED* | +| an internal whitespace run collapsed | **matched** | *UNVERIFIED* | +| a **slash** where the master has a **space** | **matched** | not sent | +| an **en dash** where the master has a space | **rejected** | *UNVERIFIED* | +| an **underscore** where the master has a space | **rejected** | *UNVERIFIED* | +| `AND` for `&` | **rejected** | rejected | +| a **missing** suffix word | **rejected** | rejected | +| an **added** suffix word | **rejected** | not sent | +| **NFD** against an NFC master | **rejected** | not sent | + +**One row here was mislabelled and is corrected.** The first run of this probe recorded `AND` for +`&` as rejected, but what it actually sent was a name with `AND CO` **appended** — against a master +carrying no `&` at all. That measures an added suffix, not a substitution, and the label was wrong +even though the verdict happened to be. It was re-run against `Profit & Loss A/c`, a reserved +ledger present in every company: + +| supplied against live `Profit & Loss A/c` | result | +| --- | --- | +| `Profit & Loss A/c` | **matched** — control | +| `profit & loss a/c` | **matched** — case folds on a name carrying `&` and `/` | +| `Profit AND Loss A/c` | **rejected** — the substitution, now measured here | +| `profit and loss a/c` | **rejected** | +| `Profit & Loss` | **rejected** — a missing suffix word | +| `Profit & Loss A/c AND CO` | **rejected** — an added suffix word, what the first run really sent | + +So §9.4b's abbreviation findings hold on licensed 7.1 as well, and this section now says which +of them it measured rather than which it meant to. + +**Composition was measured separately, because twelve single-axis results do not license it.** +Each row above is **one** transformation away from exact, so together they say each transformation +works alone and nothing about applying several at once — which is exactly what any fold does. Two +reviewers raised that independently, and it was worth a second run rather than an argument. Eight +more variants, same method, same readback and deletion: + +| supplied | axes stacked | result | +| --- | --- | --- | +| `MB PILOT ALPHA (5550001001)` | control | **matched** | +| ` mb pilot alpha (5550001001) ` | case + leading + trailing | **matched** | +| `mb-pilot-alpha-(5550001001)` | case + hyphen-for-space | **matched** | +| ` mb-pilot-alpha-(5550001001) ` | case + hyphen + leading + trailing | **matched** | +| `MB/PILOT ALPHA (5550001001)` | slash + collapsed run | **matched** | +| `mb-pilot alpha/(5550001001)` | case + hyphen + slash, mixed in one name | **matched** | +| ` mb-pilot/alpha (5550001001) ` | all five at once | **matched** | +| ` mb probe ledger a ` against `MB-PROBE-LEDGER-A` | case + space-for-hyphen + surrounding + run | **matched** | + +All eight posted against the intended master, confirmed by day-book readback. **So the folds +compose**, and a canonical form applying every measured transformation before comparing is +licensed by measurement rather than by extrapolation from the single-axis rows. + +**What this says.** On licensed 7.1, Tally treats **space, hyphen and slash** as interchangeable +separators, collapses internal whitespace runs, ignores leading and trailing whitespace, folds +**ASCII** case, and is otherwise **exact on codepoints**. + +> **RULE: separators fold, and the set is `space`, `-`, `/` — nothing else.** An en dash and an +> underscore are ordinary characters to Tally and are **not** separators, so a fold that treats +> "punctuation" or "separators" as a class is wider than the gateway and will merge masters it +> keeps apart. + +That is the trap §9.4b warned about, arriving from the other side: the danger was never only that +a reader would fold too much, it was that "normalises separators" names no particular set. Two of +the four separators tested are folded and two are not, and nothing about their appearance predicts +which. + +**Canonical equivalence is still refused**, consistent with the exact-codepoint finding recorded +elsewhere in this document: an NFD spelling of an NFC ledger does not resolve. A fold that +normalises before comparing merges masters this gateway keeps apart. + +**Scope.** One instance, one build, one licence tier, **ledgers only**, one company, and the +measurement is of *import-time* name resolution — not collection filters, not stock items, groups +or voucher types, and not voucher numbers. §9.4b's Educational scope stands as its own row; this +does not retire it, and where the two disagree they disagree about different SKUs rather than +about the same one. + +**Fixtures.** `MB-PROBE-LEDGER-A` and `MB CAFÉ PROBE` remain in `BRIDGE CORPUS OPENING` under +`Suspense A/c`, carrying no balances, so this is repeatable. They post-date the catalogue digest +recorded in `TEST_CORPUS.md` §9.2. + + +### 9.4c Real catalogues carry families a partial name cannot separate + +**VERIFIED 2026-09-10** for the counts, across 16 loaded companies on both lab instances; the rule +built on them is PARTIAL. `TEST_CORPUS.md` §9.1 carries the procedure, the per-company figures and +what they do not cover. + +Live books name parties in **sequentially-numbered families** — one observed catalogue runs a single +prefix across more than a hundred ledgers that differ only in a trailing number. A source name that +is a truncation of one of them reaches the whole family and distinguishes no member of it. + +**Why that is a protocol-level fact and not an implementation detail:** any client matching a +supplied name against a read catalogue meets it, and the tempting response — offer the first N and +let a human pick — is measured wrong. Listing an arbitrary capped slice of such a family **put the +intended master outside the offered list about a third of the time** (present in 65.6% of lists, +against 100% once families beyond the cap were withheld and counted instead). + +> **RULE: where a supplied name reaches a family it does not separate, report the count and withhold +> the list. An arbitrary slice of a family is not a shortlist — it is a wrong answer that looks like +> a shortlist.** + +The scope is narrow and matters: the catalogue side is live, and every *source* name in the +measurement is a fabricated mutation of a live name. It measures the rule against real naming +habits, not against real operator input. + + ### 9.5 Identity after write **VERIFIED.** `LASTMID` is **0** on successful master creates — unusable for master identity; diff --git a/docs/tally/TEST_CORPUS.md b/docs/tally/TEST_CORPUS.md index 3476fd597..66f63494e 100644 --- a/docs/tally/TEST_CORPUS.md +++ b/docs/tally/TEST_CORPUS.md @@ -349,6 +349,164 @@ bytes, so it cannot support any claim about the exact bytes a real instance rece --- +## 9. Master-binding ledgers in `BRIDGE CORPUS OPENING` + +**VERIFIED 2026-09-10** for the seeding and the coverage counts; the binding behaviour built on +them is **PARTIAL**. Scope of each, so neither is read for more than it covers: + +| claim | confidence | what establishes it | +| --- | --- | --- | +| The ten ledgers exist in that company and nowhere else | **VERIFIED** | `CREATED=10, ALTERED=0, ERRORS=0`, then a readback of the ledger list naming all ten, plus a readback of a guard company showing none | +| No book carried an embedded **numeric** identifier before this | **VERIFIED** | all 16 loaded companies read through the `StandardLedgerCatalogV1` request, responses written to files and parsed from the files; 470 names, **0 numeric**. Exactly **one** name yielded a *code* identifier, so the absence is of the numeric shape only — stated narrowly because these counts are what justify the rule's strictness | +| The identifier rule behaves correctly against live-read names | **PARTIAL** | exercised against these ten seeded names only, on one instance, one licence tier, one Tally build. Fabricated *source* names against live *catalogue* names — no real source document has been bound end to end | +| The shipped consumer path runs end to end against a real instance | **VERIFIED 2026-09-11**, all ten rows | the branch's own `bridge_mcp` binary driven over stdio against licensed TallyPrime 7.1 Silver, reading this company's catalogue over the wire and returning the binder's report. Run twice — before and after the resolving fold was narrowed — against the same catalogue digest; see §9.2 | +| Binding is safe on catalogues generally | **UNVERIFIED** | one company, one instance, one Tally build, and every *source* name fabricated. No engagement has run a real document through this path; the mutation sweep is fabricated mutations of live names, not observed operator input | + +**Added 2026-09-10.** Ten ledgers prefixed `MB `, seeded so the master-binding +identifier rule has live coverage. Before this, across 470 live ledger names read from all +16 loaded companies, **not one yielded a numeric identifier and exactly one yielded a code +identifier** — so the numeric rule had no live coverage at all and the code rule had a +single instance. The rule that distinguishes `bridge_tally_core::master_binding` from fuzzy +matching was otherwise qualified by fabricated data alone. + +| ledger | what it exercises | +| --- | --- | +| `MB PILOT ALPHA (5550001001)` | a unique embedded number | +| `MB PARTY BETA (5550001002)`, `MB PARTY GAMMA (5550001003)` | the same, for name-vs-identifier cases | +| `MB PARTY DELTA (5550001009)`, `MB PARTY EPSILON (5550001009)` | **two masters sharing one identifier** — must refuse, never bind | +| `MB ITEM PH01AB00` | a code identifier, matched across punctuation | +| `MB PURCHASES FY2025`, `MB SALES FY2025` | a shared fiscal-period label that must **not** be treated as an identifier | +| `MB TRADING COMPANY`, `MB TRADING COMPANY LIMITED` | a truncation / near-duplicate pair | + +**Chosen company.** `BRIDGE CORPUS OPENING` (GUID `915d42f8-42ae-4b03-8291-55f596e3a2ea`), +because it verifies as a single identity tuple and had only eight ledgers. **Not** +`BRIDGE PROBE B SANDBOX`, despite that being where corpus manufacturing was first proven: +it and `BRIDGE PROBE B SANDBOX - (from 1-Apr-26)` share one GUID, and Bridge's own read +path refuses that company with `company_identity_ambiguous`. Do not write to it by name. + +**Blast radius, deliberately small.** All ten are parented to `Suspense A/c`, which is not +a party group, so receivable/payable and ageing measurements on this book are unaffected. +They carry no opening balance and no vouchers. Every name is prefixed `MB `, so they are +trivially identifiable and removable. Master `AlterID` for this company did move; anything +pinning `ALTMSTID` for `BRIDGE CORPUS OPENING` predates 2026-09-10. + +**Import method.** `REPORTNAME=All Masters`, `ACTION="Create"`, one pilot ledger sent and +verified in the intended company *and confirmed absent from a guard company* before the +remaining nine. Counters were `CREATED=10, ALTERED=0, ERRORS=0`, and every name was +confirmed by a readback of the ledger list — counters alone prove nothing, since Tally +rewrites imports silently. **Do not re-send the create file:** an identical `Create` is a +silent `Alter` that overwrites. + +### 9.1 Candidate quality across sixteen live catalogues, 2026-09-10 + +**VERIFIED for the counts; the rule they justify is PARTIAL.** Recorded here because the binder +encodes this result in two comments and a status name, and a measurement that lives only in an +implementation comment cannot be audited (P6, P9). + +**Procedure.** Every ledger name of all 16 loaded companies was read through +`StandardLedgerCatalogV1`, responses written to files and parsed from the files. 434 fabricated +source names — mutations of those live names — were bound against their own company's catalogue, +and each result was checked for whether the master the mutation came from appeared at all. + +| | listing a capped slice of a prefix family | withholding a family over `MAX_PREFIX_FAMILY` | +| --- | --- | --- | +| intended master present in the candidate list | **65.6%** | **100%** (434 of 434) | +| median candidates offered per source name | **40** | **2** | +| share of the catalogue offered | **62.8%** | — | + +**What drove it.** `CatalogPrefix` produced 12,108 of 12,793 candidates, almost all from +sequentially-numbered party families — a truncated `DN Party 0` reaches `DN Party 001`…`120` and +separates none of them. Capping the list at `MAX_CANDIDATES_PER_ENTITY` then printed an arbitrary +25 of them, and the arbitrariness is the defect: **the intended master was absent from about a +third of the lists.** So a family over the bound is counted and deliberately not listed, which is +what `NoDiscriminatingCandidate` means. + +**Scope, so this is not read for more than it covers.** Two instances, 16 companies, one Tally +build. The catalogue side is live; every *source* name is a fabricated mutation, so this measures +how the rule behaves against real naming habits, not against real operator input. The 100% is a +property of this corpus and these mutations, not a guarantee. It says the withholding rule fixed +the failure it was written for; it does not say candidate lists are sufficient in general. + +--- + +### 9.2 The end-to-end slice, 2026-09-11 + +**VERIFIED.** The reviewable claim before this was that the binder had never run through the +surface that ships it: the crate had tests, and the catalogue side had live coverage, but no +run had gone request-to-report through the consumer. This one does. + +**Procedure.** `cargo build --bin bridge_mcp`, first at commit `9e34cd77` and again at +`330bd696` after the resolving fold was narrowed, then the binary driven +over stdio with a real MCP session — `initialize`, `notifications/initialized`, +`tools/call validate_masters` — against `http://127.0.0.1:9001`, TallyPrime 7.1, licence tier +**silver**, `education_mode=false`. The tool read this company's ledger catalogue over the wire +(89,576 bytes, `evidence.state=complete`) and returned the binder's report. Ten fabricated source +names, chosen so that every decision the binder can reach is reached against **live-read master +names**. + +| requested (fabricated) | returned | what it establishes | +| --- | --- | --- | +| `MB PILOT ALPHA (5550001001)` | `exact` | byte equality | +| `mb pilot alpha (5550001001)` | `identifier` | the identifier is consulted **before** the name, so a case variant never reached the fold | +| `MB-PILOT-ALPHA-(5550001001)` | `near_miss`, 1 candidate, `normalized_equal` | the unverified separator direction **suggests and does not resolve** — it returned `normalized` before the fold was narrowed | +| `Alpha Pilot Account 5550001001` | `identifier` | **the rule this module exists for**: a name sharing no word with the master bound on its embedded number | +| `Zeta Holdings 5550001009` | `near_miss`, 2 candidates, `shared_identifier` | one identifier on two masters refuses and shows both | +| `MB TRADING COMPANY LTD` | `near_miss`, 2 candidates | a truncation surfaces both neighbours and chooses neither | +| `MB EXPENSES FY2025` | `near_miss`, 2 candidates, `shared_token` | a shared period label did **not** become an identifier | +| `MB ITEM PH-01-AB-00` | `identifier` | a code matched across punctuation | +| `Zeta Nowhere Traders` | `missing`, `listing: "none"` | an absence stated as an absence, not as an empty list | +| `MB PARTY BETA` | `near_miss`, 4 candidates | a prefix family surfaced whole | + +**`exact_live_spelling` appeared on bound rows only** — `exact`, `identifier`, `normalized` — +and on no refusal. The same ten names sent to the **previously installed** server, same company +and instance minutes earlier, returned `exact_live_spelling` alongside `match_state: "near_miss"` +for two of them. That field on a refusal is a guess wearing the shape of an answer, and it is +what this change deletes; the two runs are the before and after on one real instance. + +**What this slice does not establish.** One company, one instance, one Tally build, one licence +tier. Every *source* name is fabricated — a real source document has still never been bound, so +nothing here speaks to how operator-written names actually differ from master names. It exercises +the MCP consumer; the desktop consumer shares the crate but was not driven. + +**Superseded in part, 2026-09-12.** The separator row below was read as a defect and drove a +narrowing of the resolving fold. `TALLY_PROTOCOL_REFERENCE.md` §9.4d then measured that same +equivalence directly on **licensed** TallyPrime 7.1 and found Tally does accept it — along with +leading whitespace, collapsed runs and a slash — so the fold was widened back to what the gateway +actually does. Read the paragraph below as the history it is: the row's `normalized` result was +right, and the reasoning that called it wrong was working from §9.4b's Educational scope. + +**It found a defect, which is the reason to run these.** On the first run the third row *bound* +`MB-PILOT-ALPHA-(5550001001)` to a master carrying spaces. That is the **reverse** of the +direction `TALLY_PROTOCOL_REFERENCE.md` §9.4b measured, and §9.4b marks it **UNVERIFIED** — so +the binder was resolving on evidence that does not support resolving, which is exactly what +§9.4b exists to prevent. Nothing in the unit suite said so, because the suite encoded the same +assumption the implementation did. + +**The second run, after the narrowing, is what closes it.** Same ten names, same company, same +catalogue — `catalogue_evidence_sha256` `0767077c…` on both runs, so the book did not move +underneath the comparison. Row three now returns a near-miss carrying +`MB PILOT ALPHA (5550001001)` as its **sole** candidate under `normalized_equal`, and reports the +code it could not resolve on. That is the whole intent of the two-fold split in one row: the +looser fold still reaches the master, and no longer answers for it. The other nine rows returned +identically across both runs. + +So this table is VERIFIED for all ten rows, at `330bd696`. ADR 0016 §3 records the narrowing and +its cost. + +**The catalogue digest has since moved, and a mismatch is not drift.** Qualifying §9.4d needed two +ledgers — `MB-PROBE-LEDGER-A` and `MB CAFÉ PROBE` — and they remain in this company under +`Suspense A/c` carrying no balances, so that the measurement is repeatable. Both post-date the +runs above, so `catalogue_evidence_sha256` for `BRIDGE CORPUS OPENING` no longer equals +`0767077c…`. Anyone re-running this slice should expect a different digest and check the ledger +list before treating it as the book changing underneath them. + +**What it found within minutes.** The `DELTA`/`EPSILON` pair exposed a defect no fabricated +fixture had produced: a *byte-exact* request for `MB PARTY DELTA (5550001009)` was being +refused as `IdentifierConflict`, because the number in its name is shared. That made the +ledger permanently unimportable, since the write gate admits `exact` only. Byte equality is +now decisive over an ambiguous identifier; only a *decisive* identifier pointing elsewhere +outranks an exact name. + ## 6. Changelog | Date | Change | diff --git a/docs/tally/compatibility/compatibility-matrix.json b/docs/tally/compatibility/compatibility-matrix.json index 52b02793a..ddecfadfa 100644 --- a/docs/tally/compatibility/compatibility-matrix.json +++ b/docs/tally/compatibility/compatibility-matrix.json @@ -1,7 +1,7 @@ { "schema_version": 1, "bridge_commit_sha": "be1c20cc3fd66fa1ece196505c69f26e555e4b8e", - "compatibility_surface_sha256": "1a2641e82097ff7e70ed7f68c7c142da57606cfcff3eee1ceaf86c9b0ac9522b", + "compatibility_surface_sha256": "e7263d3dd4bdbdbc8b42fd1685a0cd3c25a29ae71e25df5740deeb73f2701c57", "claims": [ { "claim_id": "erp9-6-6-3-windows-education-xml-one-company", diff --git a/docs/tally/compatibility/compatibility-surface.json b/docs/tally/compatibility/compatibility-surface.json index 922dd9ce9..f351d5377 100644 --- a/docs/tally/compatibility/compatibility-surface.json +++ b/docs/tally/compatibility/compatibility-surface.json @@ -35,7 +35,7 @@ }, { "path": "docs/tally/TALLY_PROTOCOL_REFERENCE.md", - "sha256": "9bae024f2a7a423d999ecc911953b1e13872f2f585f7fdfdf3e777034d9fe737" + "sha256": "724cf4ae4c34ae39858959eb3da8ca488b740db8c5234a651d0574228ae5bbcf" }, { "path": "docs/tally/compatibility/README.md", @@ -123,15 +123,15 @@ }, { "path": "src-tauri/Cargo.lock", - "sha256": "929a359a24c809dff87405f8103fe648ae60dba0aaba3229911ec694683708d7" + "sha256": "1b6484e09fa0cc08355dfc0cccda5abbbb4651426081401ab63caccf372b5245" }, { "path": "src-tauri/Cargo.toml", - "sha256": "5cfe6c1fba7b20dd7d7a65ffc96039f41be9d7d55129abe071f470d0b10baf5a" + "sha256": "d4071736b6a6e5cc10f8252c36c0de2cfc48de6c01a03da8e0cb95dfeed240f2" }, { "path": "src-tauri/crates/bridge-tally-core/Cargo.toml", - "sha256": "67868c232e5fcb21b8e0098732b8cb92f87353f90460cf21090feeb262ca1c31" + "sha256": "512cfa03c1a126c36433b6de54a9fabdd822d8d1ac5db8f8d3e2ef5e7ec370e0" }, { "path": "src-tauri/crates/bridge-tally-core/src/bills_reconciliation.rs", @@ -139,7 +139,11 @@ }, { "path": "src-tauri/crates/bridge-tally-core/src/lib.rs", - "sha256": "fe515176a64d96322b49843b96facfbe51c71e320ffbe03c36a8cf1860fe8249" + "sha256": "58674602eb3131c101ace7638d43cf550d74b29eb3848b0c908687bb2c9fbf9c" + }, + { + "path": "src-tauri/crates/bridge-tally-core/src/master_binding.rs", + "sha256": "fee1d066c63446dbadf4efb7a9b795399899b0b7780cd8ad2a234de4b2f1dbf0" }, { "path": "src-tauri/crates/bridge-tally-core/src/pack_models.rs", @@ -327,7 +331,7 @@ }, { "path": "src-tauri/src/agent_import.rs", - "sha256": "2390824cffe0596d13e8956d30436f2482c99a5d101ff3b53d217553e5ee3cca" + "sha256": "c186d5d8618ce1b92ff02cf4451abf5e76eb435647e3c9ecae6e5aaea6210ba5" }, { "path": "src-tauri/src/agent_ledgers.rs", @@ -579,7 +583,7 @@ }, { "path": "src-tauri/src/source_draft/catalog.rs", - "sha256": "bf583c935b7668a9be3420741b6709b4c8bad42bd4a781317d2a9b8e417c4be5" + "sha256": "5a6300839e7871c9ed813b12ff1d941d91e832ccd3dfb7fc7f161be262d3311b" }, { "path": "src-tauri/src/source_draft/files.rs", @@ -775,7 +779,7 @@ }, { "path": "src/source-draft-types.ts", - "sha256": "d188ea34e3bd01e6be54927f13bf1d3820a37d8ae46eaf4ddfe93627806e99f9" + "sha256": "cc7885a1f6942a63a7768f207a21198b94e601c3413fa3600f22ac05b5664939" }, { "path": "src/source-draft.css", @@ -799,7 +803,7 @@ }, { "path": "tools/Cargo.lock", - "sha256": "b68a1a0d5c735459b7280657ced1b7d426039266e361ec4d75b17b933bd1e785" + "sha256": "62d922fb0c6b8b7fe1313bfb9058f1991760bfd04bfc5e28552dc4ec9ef2e11a" }, { "path": "tools/bridge-tally-compatibility/Cargo.toml", @@ -811,7 +815,7 @@ }, { "path": "tools/bridge-tally-compatibility/src/lib.rs", - "sha256": "442c3ec2f4a4e941db93fe44b6d12035544154c3eeb7ea1344c9200a28572c35" + "sha256": "91b726ce49d93068e8d6c16395e4c629d84c1800927245c65dc4aaea26ac4501" }, { "path": "tools/bridge-tally-compatibility/src/main.rs", @@ -846,5 +850,5 @@ "sha256": "a8ac2714fecf51947f2822c8c46d7ce2e8602c732780ff60566a7771f0836f9a" } ], - "manifest_sha256": "1a2641e82097ff7e70ed7f68c7c142da57606cfcff3eee1ceaf86c9b0ac9522b" + "manifest_sha256": "e7263d3dd4bdbdbc8b42fd1685a0cd3c25a29ae71e25df5740deeb73f2701c57" } \ No newline at end of file diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index f4061eb4e..a8cc6ac57 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -441,7 +441,6 @@ dependencies = [ "tokio-util", "tracing", "tracing-subscriber", - "unicode-normalization", "uuid", "windows-sys 0.61.2", "x509-parser", @@ -460,6 +459,7 @@ dependencies = [ "sha2 0.11.0", "thiserror 2.0.20", "tokio", + "unicode-normalization", ] [[package]] diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index e34931679..1a7335268 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -89,7 +89,6 @@ tokio-util = { version = "0.7", features = ["io"] } tracing = "0.1" tracing-subscriber = { version = "0.3", features = ["env-filter"] } uuid = { version = "1", features = ["v4", "serde"] } -unicode-normalization = "0.1" x509-parser = "0.18" zeroize = "1" pdf-writer = "0.15.0" diff --git a/src-tauri/crates/bridge-tally-core/Cargo.toml b/src-tauri/crates/bridge-tally-core/Cargo.toml index 09a5cbb71..d3873e4ce 100644 --- a/src-tauri/crates/bridge-tally-core/Cargo.toml +++ b/src-tauri/crates/bridge-tally-core/Cargo.toml @@ -15,6 +15,7 @@ serde = { version = "1", features = ["derive"] } serde_json = "1" sha2 = "0.11" thiserror = "2" +unicode-normalization = "0.1" [dev-dependencies] tokio = { version = "1", features = ["macros", "rt-multi-thread"] } diff --git a/src-tauri/crates/bridge-tally-core/src/lib.rs b/src-tauri/crates/bridge-tally-core/src/lib.rs index f45624f90..9860d52de 100644 --- a/src-tauri/crates/bridge-tally-core/src/lib.rs +++ b/src-tauri/crates/bridge-tally-core/src/lib.rs @@ -8,6 +8,7 @@ pub use bridge_tally_primitives::{ }; pub mod bills_reconciliation; +pub mod master_binding; mod pack_models; pub mod reconciliation; pub mod report_tie_out; diff --git a/src-tauri/crates/bridge-tally-core/src/master_binding.rs b/src-tauri/crates/bridge-tally-core/src/master_binding.rs new file mode 100644 index 000000000..a7d732ad5 --- /dev/null +++ b/src-tauri/crates/bridge-tally-core/src/master_binding.rs @@ -0,0 +1,2070 @@ +//! Deterministic binding of source-document entity names to one company's +//! observed masters. +//! +//! See `docs/adr/0016-master-binding-authority.md`. Three rules carry the whole +//! contract: an embedded identifier is matched before any name, nothing binds +//! unless it is unique on both sides, and a near-miss is never resolved — it is +//! reported with its candidates so an operator decides. +//! +//! This module performs no I/O, holds no company identity, and calls no model. +//! A returned binding is a proposal: a caller that intends to act on one +//! re-reads the catalog and revalidates the selection through the admission +//! path that owns identity. + +use std::collections::{BTreeMap, BTreeSet}; + +use serde::{Deserialize, Serialize}; +use unicode_normalization::UnicodeNormalization; + +/// Most masters one catalog may carry. +pub const MAX_CATALOG_ENTRIES: usize = 20_000; +/// The aggregate a catalog's names may occupy, independent of how they divide +/// into entries. Eight mebibytes is far above any observed book — the largest +/// catalogue read here is 470 names — and far below the 327 MB the per-name and +/// per-entry bounds admit between them. +const MAX_CATALOG_NAME_BYTES: usize = 8 * 1024 * 1024; +/// Most entities one binding request may name. This must stay at or above what +/// a consumer's own parser admits: Bridge's source-draft parser accepts 2,000 +/// vouchers of 20 entries, and a bound below that turned a valid draft into a +/// silently empty binding result. +pub const MAX_SOURCE_ENTITIES: usize = 40_000; +/// The aggregate a request's source names may occupy, as `MAX_CATALOG_NAME_BYTES` +/// is for the catalog side. The count and per-name bounds do not bound their +/// product any better here than there: 40,000 names of 16,384 characters +/// satisfies both and is 2.6 GB of names alone, before the two folds each +/// entity retains beside its name and the clones this module makes of them. +/// +/// Sixteen mebibytes rather than the catalog's eight, because a request may +/// legitimately name one master many times over — the same ledger on 2,000 +/// vouchers — where a catalog may not. +const MAX_SOURCE_NAME_BYTES: usize = 16 * 1024 * 1024; +/// Longest accepted master or source name, in characters. This bounds +/// pathological input; it is not a claim about what Tally accepts, and a +/// caller with a stricter contract of its own enforces that at its own +/// boundary. +pub const MAX_NAME_CHARS: usize = 16_384; +/// Most candidates retained per unbound entity. +pub const MAX_CANDIDATES_PER_ENTITY: usize = 25; +/// Total candidate-name bytes one report may allocate, across all entities. +/// +/// A per-entity cap does not bound a report: a draft the source parser admits +/// can carry tens of thousands of entries that each list 25 long names, and the +/// clones exist the moment the report is built. A consumer capping its own copy +/// afterwards bounds only the second copy. This is spent in entity order; +/// entities past it report their true `candidate_count` with no candidates +/// listed and truncation flagged. +pub const MAX_REPORT_CANDIDATE_BYTES: usize = 256 * 1024; +/// Most identifiers one name may carry. Exceeding it is refused, never +/// truncated. +pub const MAX_IDENTIFIERS_PER_NAME: usize = 32; +/// Digits a numeric run needs before it is treated as an identifier. Eight +/// excludes a year, a rate, a house number and a masked last-four; a mobile, +/// an account number and a customer code all clear it. +pub const MIN_NUMERIC_IDENTIFIER_DIGITS: usize = 8; +/// The longest run that can still be somebody's identifier. +/// +/// This bounds `retained_tag`, which is the point. An unresolved entity carries +/// its identifiers into a fallback so an operator can find the money later, and +/// the documented way to carry them is a narration — which `agent_import` +/// refuses over 2,000 characters. Unbounded values made `assign_fallback` +/// succeed while producing a readback identity that could not be written, which +/// is a worse failure than refusing: it is discovered at the write, not here. +/// +/// Bounding the *value* rather than truncating the tag keeps the tag complete. +/// Thirty-two identifiers at this length, with their `kind:` prefixes and +/// separators, stay under that narration limit. A run longer than this is not +/// an account number, a registration or a part code; it is a digit sequence +/// that happens to be long, and treating it as an identity was never right. +const MAX_IDENTIFIER_CHARS: usize = 48; +/// Alphanumeric characters a mixed letter-and-digit token needs before it is +/// treated as a code identifier. +/// +/// Eight, raised twice under review. Enumerating the period shapes that must +/// not be identifiers — `FY25`, then `APR2025`, then `2025Q1` — is a losing +/// game, and each miss binds two unrelated ledgers that merely share a period. +/// Requiring real length is the rule that does not depend on having thought of +/// every label: a part number or registration code clears it, and a period +/// label does not. Measured against 485 live ledger names, exactly one yields a +/// code identifier at all, so this costs nothing observed. +pub const MIN_CODE_IDENTIFIER_CHARS: usize = 8; +/// Digits a code identifier needs alongside at least two letters. +pub const MIN_CODE_IDENTIFIER_DIGITS: usize = 3; +/// Shortest comparison key that may take part in a prefix near-miss. +pub const MIN_PREFIX_KEY_CHARS: usize = 3; +/// Shortest token that may take part in a shared-token near-miss. +pub const MIN_TOKEN_CHARS: usize = 3; +/// Masters a single prefix may match before the prefix stops discriminating. +/// Beyond this the match is a name *family*, and an arbitrary slice of it is +/// worse than saying so. +pub const MAX_PREFIX_FAMILY: usize = MAX_CANDIDATES_PER_ENTITY; +/// Share of the catalog above which a token stops discriminating. +pub const COMMON_TOKEN_PERCENT: usize = 10; +/// Catalog size below which no token is treated as common. +pub const COMMON_TOKEN_MIN_CATALOG: usize = 20; + +/// The master class a catalog and a report belong to. Both classes have failed +/// in practice and the rules are identical for both; the class is carried so a +/// report cannot be applied against the wrong catalog. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum MasterClass { + Ledger, + StockItem, +} + +/// Binding refuses rather than degrades. Every variant is a fail-closed +/// boundary check on input that was never observed, never usable, or already +/// undecidable before any matching ran. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +pub enum MasterBindingError { + /// Binding against a book that was never read is the failure this whole + /// contract exists to prevent. It is an error, not an empty report. + #[error("master catalog was empty")] + CatalogEmpty, + /// Two masters carry byte-identical names, so a name cannot identify one. + /// Names differing only in surrounding whitespace are *not* duplicates — + /// they are retained verbatim and collide on the comparison key instead, + /// which surfaces them as an ambiguity rather than failing the read. + #[error("master catalog carried a duplicate name")] + CatalogDuplicateName, + #[error("master catalog exceeded its bound")] + CatalogTooLarge, + #[error("source entity list exceeded its bound")] + TooManySourceEntities, + /// The entity *count* is within bounds but their names together are not. + #[error("source entity names exceeded their aggregate bound")] + SourceNamesTooLarge, + #[error("name was blank")] + NameBlank, + #[error("name exceeded its bound")] + NameTooLong, + #[error("name carried a control character")] + NameUnsafe, + /// A caller-supplied identifier hint that yields no identifier would fail + /// silently, so it fails loudly instead. + #[error("identifier hint carried no usable identifier")] + IdentifierHintUnusable, + /// Keeping only the first few would discard the identifier that pointed at + /// a different master, turning a conflict into a bind. + #[error("name carried more identifiers than the bound")] + TooManyIdentifiers, + #[error("fallback master was not a current catalog entry")] + FallbackNotInCatalog, + /// A catalog of the wrong class, a catalog the report was not produced + /// from, or an entity from another report. + #[error("catalog did not match the report it is used with")] + ClassMismatch, +} + +impl MasterBindingError { + /// A stable code safe to surface to an operator or a tool result. + pub fn safe_reason_code(&self) -> &'static str { + match self { + Self::CatalogEmpty => "master_catalog_empty", + Self::CatalogDuplicateName => "master_catalog_duplicate_name", + Self::CatalogTooLarge => "master_catalog_too_large", + Self::TooManySourceEntities => "master_source_entities_too_many", + Self::SourceNamesTooLarge => "master_source_names_too_large", + Self::NameBlank => "master_name_blank", + Self::NameTooLong => "master_name_too_long", + Self::NameUnsafe => "master_name_unsafe", + Self::IdentifierHintUnusable => "master_identifier_hint_unusable", + Self::TooManyIdentifiers => "master_identifiers_too_many", + Self::FallbackNotInCatalog => "master_fallback_not_in_catalog", + Self::ClassMismatch => "master_class_mismatch", + } + } +} + +/// The shape an identifier was recognized by. Both canonicalize away the +/// punctuation an operator happened to type. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum IdentifierKind { + /// A digit run: a mobile, an account number, a numeric customer code. + Numeric, + /// A mixed letter-and-digit token: a part number, a registration code. + Code, +} + +/// One stable identifier embedded in a name, in canonical form. +#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)] +pub struct Identifier { + pub kind: IdentifierKind, + pub value: String, +} + +/// The rule that produced a candidate. There is deliberately no score: a score +/// invites a threshold, and a threshold auto-resolves the case this contract +/// exists to keep in front of a human. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum CandidateRule { + /// Byte-equal to the observed master name. Only reachable on a refusal: + /// where byte equality settles the question it binds instead, so this + /// appears exactly when something outranked it — an identifier pointing + /// elsewhere — and it is the other half of that disagreement. + ExactName, + /// Shares an embedded identifier, but the identifier was not decisive. + SharedIdentifier, + /// Equal under the comparison key, but the key was not unique. + NormalizedEqual, + /// The catalog name extends the source name — the source was truncated. + CatalogPrefix, + /// The source name extends the catalog name. + SourcePrefix, + /// Shares a token that discriminates within this catalog. + SharedToken, +} + +impl CandidateRule { + fn rank(self) -> u8 { + match self { + Self::ExactName => 0, + Self::SharedIdentifier => 1, + Self::NormalizedEqual => 2, + Self::CatalogPrefix => 3, + Self::SourcePrefix => 4, + Self::SharedToken => 5, + } + } +} + +/// A master an operator may choose, with the rule that surfaced it. No +/// candidate is marked best, and the order is rule-then-name, not similarity. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +pub struct Candidate { + pub catalog_name: String, + pub rule: CandidateRule, +} + +/// Why an entity did not bind. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum UnboundReason { + /// One embedded identifier is carried by more than one master. + IdentifierConflict, + /// An identifier and an exact name pointed at different masters. + IdentifierNameConflict, + /// More than one master shares the comparison key. + NameAmbiguous, + /// Candidates exist but none was decisive. This is the four-near-miss case + /// that rejected a batch once; a single candidate stays here too. + NearMiss, + /// The source name matches a whole family of masters and distinguishes + /// none of them — a truncated `DN Party 0` against `DN Party 001`…`120`. + /// Listing an arbitrary capped slice of such a family put the right master + /// out of view about a third of the time, so the family is counted and + /// deliberately not listed. The behaviour and its rule are + /// `TALLY_PROTOCOL_REFERENCE.md` §9.4c; `TEST_CORPUS.md` §9.1 carries the + /// counts and their scope. + NoDiscriminatingCandidate, + /// No rule produced a candidate. The master is probably missing. + NoCandidate, +} + +impl UnboundReason { + /// A stable code safe to surface to an operator or a tool result. + pub fn safe_reason_code(self) -> &'static str { + match self { + Self::IdentifierConflict => "master_binding_identifier_conflict", + Self::IdentifierNameConflict => "master_binding_identifier_name_conflict", + Self::NameAmbiguous => "master_binding_name_ambiguous", + Self::NearMiss => "master_binding_near_miss", + Self::NoDiscriminatingCandidate => "master_binding_no_discriminating_candidate", + Self::NoCandidate => "master_binding_no_candidate", + } + } +} + +/// The evidence that decided a bind. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)] +#[serde(rename_all = "snake_case")] +pub enum BindingBasis { + /// An embedded identifier unique on both sides. Checked before any name. + Identifier, + /// Byte equality with the observed master name. + ExactName, + /// Equality under the comparison key, unique in the catalog. + NormalizedName, +} + +/// The masters worth showing, and — in the variant itself — what an absence of +/// them means. +/// +/// Replaces a `Vec` plus two flags, where empty was three different facts and a +/// consumer reading `is_empty()` was wrong in two of them. That shape had +/// already been got wrong twice by different lanes; here the compiler makes +/// each case an explicit decision instead. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +#[serde(rename_all = "snake_case", tag = "listing")] +pub enum Candidates { + /// Nothing resembles this name. An absence of masters, not of information. + None, + /// Every master found, listed. + /// + /// A struct variant, not a newtype: under Serde's internally tagged + /// representation a tag cannot be merged into a sequence, and a newtype + /// here failed to serialize at runtime — on the most common unresolved + /// result, while the other three variants succeeded. + Listed { listed: Vec }, + /// More were found than could be listed — the per-entity cap, or the + /// report's aggregate byte budget. + Truncated { + listed: Vec, + found: usize, + }, + /// A family this name reaches and separates none of: counted, and + /// deliberately not listed, because an arbitrary slice of it put the right + /// master out of view about a third of the time against live books. + Withheld { found: usize }, +} + +impl Candidates { + /// The masters actually listed. Empty for `None` and `Withheld` alike, so + /// never decide anything from this alone. + pub fn listed(&self) -> &[Candidate] { + match self { + Self::None | Self::Withheld { .. } => &[], + Self::Listed { listed } | Self::Truncated { listed, .. } => listed, + } + } + + /// Masters found before any truncation or withholding. + pub fn found(&self) -> usize { + match self { + Self::None => 0, + Self::Listed { listed } => listed.len(), + Self::Truncated { found, .. } | Self::Withheld { found } => *found, + } + } + + /// Whether masters exist that are not in `listed()`. The predicate a + /// consumer needs before it may report "nothing like this is present": + /// true here means the absence of a listing is not the absence of a master. + pub fn is_incomplete(&self) -> bool { + matches!(self, Self::Truncated { .. } | Self::Withheld { .. }) + } +} + +/// What could not be bound, and why. This is the operator's work item, not an +/// error path. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +pub struct Unresolved { + pub reason: UnboundReason, + /// Identifiers extracted from the source name and from caller hints, + /// retained so a fallback posting can be reallocated later without + /// re-reading the source document. + /// + /// **This must travel in a channel that survives a read back — the + /// narration.** A client-supplied `REMOTEID` is not it: Tally overwrites + /// the attribute with its own value, so a key written there cannot be + /// observed afterwards and cannot identify what to reallocate + /// (`docs/tally/TALLY_PROTOCOL_REFERENCE.md` §9.3, which records that the + /// attribute does not echo the client key on readback). + /// A parked amount whose identity went into a write-only field is + /// unreallocatable, and nothing about the write would say so. + pub unresolved_identity: Vec, + pub candidates: Candidates, +} + +/// Exactly one outcome per source entity. +/// +/// **What a `Bound` does not establish**, written here because a computed check +/// gets read for more than it covers, and the caller cannot see the gap from +/// the value alone: +/// +/// - **Not that the master still exists.** The catalog is a snapshot. A caller +/// acting on a binding re-reads and revalidates through the admission path +/// that owns identity; nothing here is a lease on the book. +/// - **Not that the name may be written as given.** Only `ExactName` is byte +/// equality. A `NormalizedName` or `Identifier` bind means the payload and +/// the live name *differ*, and Bridge's write gate admits `exact` only — use +/// `catalog_name`, not what was requested. +/// - **Not that this is the right master in business terms.** It establishes +/// that one deterministic rule selected one master uniquely. Whether that +/// party is the one the document meant is a judgement the rules cannot make. +/// - **Not any authority.** A binding is a proposal: it approves nothing, +/// creates nothing, and dispatches nothing. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +#[serde(rename_all = "snake_case", tag = "status")] +pub enum BindingStatus { + Bound { + catalog_name: String, + basis: BindingBasis, + }, + /// More than one master is defensible. + Ambiguous(Unresolved), + /// No master is defensible. + Unmatched(Unresolved), +} + +/// One source entity and its outcome. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] +pub struct EntityBinding { + pub position: usize, + /// The name exactly as the source document gave it. + pub source_name: String, + #[serde(flatten)] + pub status: BindingStatus, +} + +impl EntityBinding { + pub fn bound_name(&self) -> Option<&str> { + match &self.status { + BindingStatus::Bound { catalog_name, .. } => Some(catalog_name.as_str()), + _ => None, + } + } + + pub fn unresolved(&self) -> Option<&Unresolved> { + match &self.status { + BindingStatus::Ambiguous(unresolved) | BindingStatus::Unmatched(unresolved) => { + Some(unresolved) + } + BindingStatus::Bound { .. } => None, + } + } +} + +/// Control totals for one run. `requested == bound + unbound` and +/// `unbound == ambiguous + unmatched` always hold. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)] +pub struct BindingTotals { + pub requested: usize, + pub bound: usize, + pub unbound: usize, + pub ambiguous: usize, + pub unmatched: usize, +} + +/// The result of one binding run. +/// +/// Serializes but does **not** deserialize, and the asymmetry is the point. +/// `assign_fallback` proves provenance by comparing the catalog fingerprint it +/// recorded, and `catalog.fingerprint()` is public — so a derived `Deserialize` +/// would let any caller restore an invented `Ambiguous` entity carrying the +/// right fingerprint and draw a fallback for an entity the binder never emitted. +/// `bind` is the only way to obtain one. A report that must cross a process +/// boundary needs a restore that re-derives these fields, not a derive that +/// trusts them. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct BindingReport { + class: MasterClass, + catalog: CatalogFingerprint, + entities: Vec, +} + +impl BindingReport { + pub fn class(&self) -> MasterClass { + self.class + } + + /// The catalog this report was produced from. + pub fn catalog(&self) -> CatalogFingerprint { + self.catalog + } + + pub fn entities(&self) -> &[EntityBinding] { + &self.entities + } + + /// Everything that bound. + pub fn bound(&self) -> impl Iterator { + self.entities + .iter() + .filter(|entity| matches!(entity.status, BindingStatus::Bound { .. })) + } + + /// Everything that did not — the product of this module. + pub fn unbound(&self) -> impl Iterator { + self.entities + .iter() + .filter(|entity| !matches!(entity.status, BindingStatus::Bound { .. })) + } + + /// Parks one of *this report's* unbound entities against a fallback master + /// drawn from a catalog of the same class. + /// + /// Taking an index rather than an `EntityBinding` is the point: an entity + /// from another report — a stock-item binding, say — cannot be handed to a + /// ledger catalog, because it cannot be named here at all. The class is + /// then checked as well, so a same-shaped catalog of the wrong class is + /// refused rather than silently accepted, and the result carries the class + /// forward for anything downstream that needs to prove it. + pub fn assign_fallback( + &self, + entity_index: usize, + catalog: &MasterCatalog, + fallback_name: &str, + ) -> Result { + // Class alone is not provenance: two ledger catalogs are both + // `Ledger`, and a fallback drawn from the one the report never saw + // would name a master that was never a candidate for this entity. + if catalog.class != self.class || catalog.fingerprint != self.catalog { + return Err(MasterBindingError::ClassMismatch); + } + let entity = self + .entities + .get(entity_index) + .ok_or(MasterBindingError::ClassMismatch)?; + let unresolved = entity + .unresolved() + .ok_or(MasterBindingError::FallbackNotInCatalog)?; + let fallback = catalog + .exact(fallback_name) + .ok_or(MasterBindingError::FallbackNotInCatalog)?; + Ok(FallbackBinding { + class: self.class, + position: entity.position, + source_name: entity.source_name.clone(), + fallback_name: fallback.to_string(), + retained: unresolved.unresolved_identity.clone(), + reason: unresolved.reason, + }) + } + + pub fn totals(&self) -> BindingTotals { + let mut totals = BindingTotals { + requested: self.entities.len(), + bound: 0, + unbound: 0, + ambiguous: 0, + unmatched: 0, + }; + for entity in &self.entities { + match entity.status { + BindingStatus::Bound { .. } => totals.bound += 1, + BindingStatus::Ambiguous(_) => { + totals.unbound += 1; + totals.ambiguous += 1; + } + BindingStatus::Unmatched(_) => { + totals.unbound += 1; + totals.unmatched += 1; + } + } + } + totals + } +} + +/// An ambiguous entity parked against a fallback master, with its unresolved +/// identity retained for later reallocation. +/// +/// Constructed only from an entity that did not bind, so rebinding something +/// that already matched is not a representable state. +/// +/// **Reallocate with a Journal moving the amount off the fallback ledger. +/// Never with `Alter`, and never with `Cancel`.** `TALLY_PROTOCOL_REFERENCE.md` +/// §9.7 measured voucher `Alter` returning `CREATED=1, ALTERED=0` and creating +/// a **duplicate with the target untouched** — four keys tested, all four +/// duplicating — and §9.6 the same for `Cancel`. The counters report success +/// either way, so the obvious correction produces exactly the double-posting a +/// parked entry exists to avoid, and says it worked. +/// +/// Re-import under the same client `REMOTEID` (§9.3) is a real correction +/// path, but reaches only vouchers Bridge itself wrote; a hand-keyed voucher +/// has no client key. This is why the retained identity travels in the +/// narration: the Journal that reallocates it is written by a human or a later +/// batch, and the narration is what either can still read. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct FallbackBinding { + class: MasterClass, + position: usize, + source_name: String, + fallback_name: String, + retained: Vec, + reason: UnboundReason, +} + +impl FallbackBinding { + /// The class of the catalog this fallback was drawn from. + pub fn class(&self) -> MasterClass { + self.class + } + + pub fn position(&self) -> usize { + self.position + } + + pub fn source_name(&self) -> &str { + &self.source_name + } + + pub fn fallback_name(&self) -> &str { + &self.fallback_name + } + + pub fn reason(&self) -> UnboundReason { + self.reason + } + + pub fn retained(&self) -> &[Identifier] { + &self.retained + } + + /// The identity to carry into a narration so the parked amount can be + /// reallocated without re-reading the source. Empty when the source name + /// carried no identifier at all — which is itself worth seeing. + pub fn retained_tag(&self) -> String { + self.retained + .iter() + .map(|identifier| { + let kind = match identifier.kind { + IdentifierKind::Numeric => "numeric", + IdentifierKind::Code => "code", + }; + format!("{kind}:{}", identifier.value) + }) + .collect::>() + .join(" ") + } +} + +/// One entity named by a source document. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct SourceEntity { + position: usize, + name: String, + /// The wide fold. Suggests; never resolves. + key: String, + /// The narrow fold. Resolves. + binding_key: String, + identifiers: Vec, +} + +impl SourceEntity { + /// Parses one source name at the boundary, extracting identifiers from it. + pub fn new(position: usize, name: &str) -> Result { + Self::with_identifier_hints(position, name, std::iter::empty::<&str>()) + } + + /// Parses one source name together with identifiers the document carries + /// outside the name — a statement's payment reference, a report's mobile + /// column. A hint that yields no identifier is refused rather than ignored. + pub fn with_identifier_hints<'a>( + position: usize, + name: &str, + hints: impl IntoIterator, + ) -> Result { + let name = validated_name(name)?; + let mut identifiers = extract_identifiers(&name)?; + for (index, hint) in hints.into_iter().enumerate() { + // Bounded here rather than after the loop: the iterator is + // caller-supplied and may be unbounded, and each turn scans a + // string and allocates. A million repeated hints deduplicate to one + // identifier, so the check below never fired while the work to + // reach it was already done. Every hint yields at least one + // identifier or is refused outright, so more hints than the + // identifier bound cannot produce a usable entity however they fold + // together — the eager bound rejects nothing the late one admitted. + if index >= MAX_IDENTIFIERS_PER_NAME { + return Err(MasterBindingError::TooManyIdentifiers); + } + // A hint is caller-supplied document text like any other name, and + // must clear the same bound before anything scans or copies it. + validate_name_bounds(hint)?; + let extracted = extract_identifiers(hint)?; + if extracted.is_empty() { + return Err(MasterBindingError::IdentifierHintUnusable); + } + identifiers.extend(extracted); + } + identifiers.sort(); + identifiers.dedup(); + if identifiers.len() > MAX_IDENTIFIERS_PER_NAME { + return Err(MasterBindingError::TooManyIdentifiers); + } + Ok(Self { + position, + key: master_identity_key(&name), + binding_key: verified_fold(&name), + name, + identifiers, + }) + } + + pub fn position(&self) -> usize { + self.position + } + + pub fn name(&self) -> &str { + &self.name + } + + pub fn identifiers(&self) -> &[Identifier] { + &self.identifiers + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +struct CatalogEntry { + name: String, + key: String, + identifiers: Vec, + tokens: BTreeSet, +} + +/// One company's observed masters of one class, indexed for binding. +/// +/// Valid by construction: `bind` cannot fail because everything that could fail +/// was decided here. +/// Which catalog a report was produced from. +/// +/// Not a security property and not a Tally identity — it distinguishes two +/// catalogs of the same class held in one process, which is the state that let +/// a fallback be drawn from a book the report never saw. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize)] +pub struct CatalogFingerprint(u64); + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct MasterCatalog { + class: MasterClass, + fingerprint: CatalogFingerprint, + entries: Vec, + by_name: BTreeMap, + by_key: BTreeMap>, + by_binding_key: BTreeMap>, + by_identifier: BTreeMap>, + by_token: BTreeMap>, + common_tokens: BTreeSet, +} + +impl MasterCatalog { + /// Parses the observed master names of one class. + /// + /// Names arrive in whatever order the book returned them; the index is + /// ordered, so a report does not depend on that order. + pub fn new(class: MasterClass, names: I) -> Result + where + I: IntoIterator, + S: AsRef, + { + let mut entries = Vec::new(); + let mut accepted_bytes = 0_usize; + let mut by_name = BTreeMap::new(); + for name in names { + if entries.len() >= MAX_CATALOG_ENTRIES { + return Err(MasterBindingError::CatalogTooLarge); + } + let name = validated_name(name.as_ref())?; + // Per-name and per-count bounds do not bound the product of the + // two: 20,000 names of 16,384 characters satisfies both and is + // 327 MB of ASCII before this constructor builds its keys, tokens + // and four indexes over them. Accumulated as the iterator is + // consumed, so a lazy catalog fails before the next name is + // retained rather than after all of them are. + accepted_bytes = accepted_bytes.saturating_add(name.len()); + if accepted_bytes > MAX_CATALOG_NAME_BYTES { + return Err(MasterBindingError::CatalogTooLarge); + } + if by_name.contains_key(&name) { + return Err(MasterBindingError::CatalogDuplicateName); + } + by_name.insert(name.clone(), entries.len()); + let key = master_identity_key(&name); + entries.push(CatalogEntry { + identifiers: extract_identifiers(&name)?, + tokens: tokens_of(&key), + key, + name, + }); + } + if entries.is_empty() { + return Err(MasterBindingError::CatalogEmpty); + } + + let mut by_key: BTreeMap> = BTreeMap::new(); + let mut by_binding_key: BTreeMap> = BTreeMap::new(); + let mut by_identifier: BTreeMap> = BTreeMap::new(); + let mut by_token: BTreeMap> = BTreeMap::new(); + for (index, entry) in entries.iter().enumerate() { + by_key.entry(entry.key.clone()).or_default().push(index); + by_binding_key + .entry(verified_fold(&entry.name)) + .or_default() + .push(index); + for identifier in &entry.identifiers { + by_identifier + .entry(identifier.clone()) + .or_default() + .push(index); + } + for token in &entry.tokens { + by_token.entry(token.clone()).or_default().push(index); + } + } + + // A token carried by a large share of the catalog says nothing about + // which master is meant. The threshold is measured from the catalog + // rather than a built-in word list, so it carries no language or + // domain assumption. + let common_tokens = if entries.len() >= COMMON_TOKEN_MIN_CATALOG { + let limit = entries.len() * COMMON_TOKEN_PERCENT / 100; + by_token + .iter() + .filter(|(_, holders)| holders.len() > limit) + .map(|(token, _)| token.clone()) + .collect() + } else { + BTreeSet::new() + }; + + // Order-independent, so the same masters read twice fingerprint alike + // however the book returned them. + let fingerprint = + CatalogFingerprint(entries.iter().fold(class as u64 + 1, |accumulated, entry| { + accumulated + ^ entry + .name + .bytes() + .fold(0xcbf2_9ce4_8422_2325_u64, |hash, byte| { + (hash ^ u64::from(byte)).wrapping_mul(0x1000_0000_01b3) + }) + })); + Ok(Self { + class, + fingerprint, + entries, + by_name, + by_key, + by_binding_key, + by_identifier, + by_token, + common_tokens, + }) + } + + pub fn class(&self) -> MasterClass { + self.class + } + + /// Which catalog this is, for a caller that must prove a later value came + /// from the same one. + pub fn fingerprint(&self) -> CatalogFingerprint { + self.fingerprint + } + + /// Masters in this catalog. Never zero: an empty catalog is refused at + /// construction, so there is no emptiness for a caller to test. + pub fn master_count(&self) -> usize { + self.entries.len() + } + + pub fn names(&self) -> impl Iterator { + self.entries.iter().map(|entry| entry.name.as_str()) + } + + /// The observed name, when it is byte-identical to a current entry. + pub fn exact(&self, name: &str) -> Option<&str> { + self.by_name + .get(name) + .map(|index| self.entries[*index].name.as_str()) + } +} + +/// Binds every source entity against the catalog. +/// +/// The entity bound is enforced here rather than left to callers: a source +/// document is untrusted input, and an unbounded variant would be a rule +/// someone has to remember. Everything else was already decided by the two +/// constructors, so this is the only way binding can fail. +pub fn bind( + catalog: &MasterCatalog, + entities: &[SourceEntity], +) -> Result { + if entities.len() > MAX_SOURCE_ENTITIES { + return Err(MasterBindingError::TooManySourceEntities); + } + // The count bound and the per-name bound do not bound their product, which + // is why the catalog constructor carries an aggregate budget — and the + // source side, which is equally untrusted, carried none. Each entity is + // individually valid at 16,384 characters, and 40,000 of them are two and a + // half gigabytes of names before this function clones a single one of them + // into a report. Refused here, at the boundary where the collection first + // becomes this module's problem, rather than part-way through building the + // report it would otherwise exhaust memory producing. + // + // Both folds are counted, not just the name: they are retained per entity + // and are the same order of size, so counting the name alone would + // under-state what has already been allocated by a factor of three. + let mut source_bytes = 0_usize; + for entity in entities { + source_bytes = source_bytes + .saturating_add(entity.name.len()) + .saturating_add(entity.key.len()) + .saturating_add(entity.binding_key.len()); + if source_bytes > MAX_SOURCE_NAME_BYTES { + return Err(MasterBindingError::SourceNamesTooLarge); + } + } + let mut budget = MAX_REPORT_CANDIDATE_BYTES; + // Which source keys actually repeat, decided before any of them is bound. + // + // A first-come cap made the memo's protection depend on **source order**: + // 1,024 distinct cheap misses at the head of a draft filled it, and the + // repeated expensive key behind them was then never cached — the stall the + // memo exists to prevent, reachable by reordering the same rows. Counting + // first removes the ordering entirely, and caches only what a second row + // will ask for again. + // + // Counted on the **whole** determinant of a memo entry, not on the name + // alone. The memo is keyed by the source key *and* the masters the + // identifiers reached, so counting `key` by itself called every hint + // variant of one name repeated: 1,024 singleton variants of `Acme Branch` + // then filled the memo with entries nothing would ask for twice, and a key + // that genuinely repeated behind them could no longer be inserted — the + // stall the memo exists to prevent, reached by a different door than the + // source-order one. + // + // The identifiers are the source-side determinant of those masters: the + // same key with the same identifiers always produces the same memo key + // against a given catalog. The converse does not hold — two different + // identifier sets can reach the same masters — so this counts no pair as + // repeated that is not, and at worst declines to cache one that is. + // + // This borrows the keys rather than cloning them, so it costs no more than + // the entity list it is counting. + let mut repeats: BTreeMap<(&str, &[Identifier]), usize> = BTreeMap::new(); + for entity in entities { + *repeats + .entry((entity.key.as_str(), entity.identifiers.as_slice())) + .or_insert(0) += 1; + } + let repeated = repeats + .into_iter() + .filter(|(_, count)| *count > 1) + .map(|(key, _)| key) + .collect::>(); + + let mut memo = SearchMemo { + seen: CandidateMemo::new(), + repeated, + }; + Ok(BindingReport { + class: catalog.class, + catalog: catalog.fingerprint, + entities: entities + .iter() + .map(|entity| bind_one(catalog, entity, &mut budget, &mut memo)) + .collect(), + }) +} + +/// What `collect_candidates` produced for one distinct source name, keyed by +/// the only two things it reads: the source key, and the masters the entity's +/// identifiers reached. +/// +/// A draft may repeat one ledger name across its rows, and the search is not +/// cheap when it does: a truncated name against a large prefix family +/// materializes and clones that whole family, once per row. The candidate byte +/// budget bounds only what is serialized afterwards. +/// +/// Capped rather than unbounded, because the memo is itself a copy of every +/// candidate list it holds — a draft of 40,000 *distinct* names would trade the +/// stall for the memory the aggregate bounds elsewhere exist to prevent. The +/// repeated-name case, which is the one that stalls, needs very few entries. +type CandidateMemo = BTreeMap<(String, BTreeSet), (Vec<(usize, CandidateRule)>, usize)>; + +/// One run's search scratch: what has already been computed, and which source +/// keys a second row will ask for again. Carried together because they are one +/// decision — whether this result is worth keeping — split across two values. +/// What the identifier pass established for one entity, before any name was +/// compared. Carried together because every consumer needs all three and the +/// awkward one — how large a family was skipped — is meaningless without the +/// other two. +struct IdentifierEvidence<'a> { + /// The master whose name the source matches byte for byte, if any. + exact: Option, + /// Every master the entity's identifiers reached. + matches: &'a BTreeSet, + /// A **lower bound** on how many masters share this entity's identifiers, + /// counted without expanding the families that were skipped: the largest + /// skipped family, plus the listed masters that are not in it. Present so a + /// withheld listing can still say how many masters are involved. + withheld_holders: usize, +} + +struct SearchMemo<'a> { + seen: CandidateMemo, + /// The (source key, identifiers) pairs a second row will ask for again. + repeated: BTreeSet<(&'a str, &'a [Identifier])>, +} + +const MAX_CANDIDATE_MEMO_ENTRIES: usize = 1_024; + +fn bind_one( + catalog: &MasterCatalog, + entity: &SourceEntity, + budget: &mut usize, + memo: &mut SearchMemo<'_>, +) -> EntityBinding { + let exact = catalog.by_name.get(&entity.name).copied(); + + // Rule one: the identifier is the key, the name is a hint. A name + // comparison on a pair that carries a decisive identifier is not merely + // weaker evidence, it is actively misleading. + // Which masters *each* identifier reached, not merely which masters were + // reached. Flattening the two loses the only fact that separates a number + // shared by several masters from several numbers pointing at different + // ones, and those need opposite answers. + let mut per_identifier: Vec> = Vec::new(); + let mut identifier_matches = BTreeSet::new(); + let mut identifier_conflict = false; + let mut large_holder_points_elsewhere = false; + // How many masters a skipped family actually held. The set is not built, + // but the *count* is the one thing a reader still needs: without it a + // withheld family reported `found() == 0` and `listing: "none"`, telling + // the operator nothing shares the identifier when hundreds do. + let mut withheld_holders = 0_usize; + // The holders of the largest skipped family, kept by reference so the count + // below can ask which listed masters are *not* in it. Nothing is cloned. + let mut largest_withheld: Option<&Vec> = None; + for identifier in &entity.identifiers { + if let Some(holders) = catalog.by_identifier.get(identifier) { + // An identifier held by more masters than a candidate list may show + // is already a conflict, and its holders are a family this entity + // does not separate — the same shape `collect_candidates` withholds + // rather than slices. Nothing downstream can use the set, so it is + // not built: a catalog where one identifier is held by 20,000 + // masters would otherwise clone 20,000 elements per source row, + // before the candidate memo is even consulted. + if holders.len() > MAX_CANDIDATES_PER_ENTITY { + identifier_conflict = true; + if holders.len() > withheld_holders { + withheld_holders = holders.len(); + largest_withheld = Some(holders); + } + // Skipping the expansion must not skip the *question* the + // expansion was asked. `identifier_points_elsewhere` needs one + // fact from this set — whether it contains the byte-exact + // master — and that is a membership test, not a + // materialization. Dropping it made the invariant + // size-dependent: a hint pointing entirely elsewhere let the + // exact name bind, but only once the family grew past the cap. + if exact.is_some_and(|index| !holders.contains(&index)) { + large_holder_points_elsewhere = true; + } + continue; + } + #[cfg(test)] + HOLDER_EXPANSIONS.with(|count| count.set(count.get() + 1)); + let reached = holders.iter().copied().collect::>(); + if reached.len() > 1 { + identifier_conflict = true; + } + identifier_matches.extend(reached.iter().copied()); + per_identifier.push(reached); + } + } + + // One family's size is not the size of their union. An entity carrying two + // identifiers — one held by thirty masters and skipped, one reaching a + // thirty-first — reported thirty, because the larger of the two counts + // ignores every master the other identifier listed. + // + // The listed masters that are *not* in the skipped family are disjoint from + // it, so adding them is sound and costs nothing but a lookup: the union is + // never built, which is the whole point of skipping. It stays a **lower + // bound** — two disjoint skipped families are still counted as the larger + // alone — and that is what `Candidates::Withheld` means. Over-counting + // would be worse than under-counting here: two identifiers can be held by + // overlapping families, so summing their sizes would state a number of + // masters that do not exist. + let withheld_holders = match largest_withheld { + Some(family) => { + // `by_identifier` is filled by pushing entry indices in ascending + // order, so each holder list is sorted and a membership test is a + // bisection rather than a scan of up to a whole catalog. + debug_assert!(family.is_sorted(), "holder lists are built in order"); + withheld_holders + + identifier_matches + .iter() + .filter(|index| family.binary_search(index).is_err()) + .count() + } + None => withheld_holders, + }; + + // An identifier shared by two masters, and an entity whose identifiers + // reach two masters, are the same refusal: the operator has a naming + // collision to see, and neither case licenses a choice. + // Byte equality with an observed master name is the strongest evidence + // there is, and it names exactly one master. An identifier that happens to + // be ambiguous does not undermine it: refusing here would make a ledger + // whose embedded number is shared with another permanently unimportable — + // the same dead end that reporting `Identifier` for an exact name created. + // Only a *decisive* identifier pointing elsewhere outranks a byte-exact + // name, and that stays a reported conflict rather than a silent choice. + // + // Found by seeding two live ledgers that share an embedded number. No + // fabricated fixture had produced the combination. + // A byte-exact name survives an identifier that is merely *shared*: that + // one identifier reached the master the name spells along with its + // siblings, and the name is what separates them. It does not survive an + // identifier that reached somewhere else entirely — that is disagreement, + // and preferring the name silently discards it. + // + // The test is per identifier, not over their union. Asking whether the + // union contains the exact master answers the shared case correctly and the + // mixed case wrongly: `ACME 11111111` with a hint reaching `BETA 22222222` + // has the exact master in the union while one identifier plainly disagrees. + let identifier_points_elsewhere = large_holder_points_elsewhere + || exact.is_some_and(|index| { + per_identifier + .iter() + .any(|reached| !reached.contains(&index)) + }); + let status = if identifier_points_elsewhere { + unresolved_status( + catalog, + entity, + UnboundReason::IdentifierNameConflict, + IdentifierEvidence { + exact, + matches: &identifier_matches, + withheld_holders, + }, + budget, + memo, + ) + } else if let Some(index) = exact { + BindingStatus::Bound { + catalog_name: catalog.entries[index].name.clone(), + basis: BindingBasis::ExactName, + } + } else if identifier_conflict || identifier_matches.len() > 1 { + // An identifier shared by two masters, and an entity whose identifiers + // reach two masters, are the same refusal: the operator has a naming + // collision to see, and neither case licenses a choice. + unresolved_status( + catalog, + entity, + UnboundReason::IdentifierConflict, + IdentifierEvidence { + exact, + matches: &identifier_matches, + withheld_holders, + }, + budget, + memo, + ) + } else if let Some(matched) = identifier_matches.iter().copied().next() { + // A decisive identifier, with no byte-exact name to outrank it. This is + // the rule that decided the case fuzzy matching got wrong. + BindingStatus::Bound { + catalog_name: catalog.entries[matched].name.clone(), + basis: BindingBasis::Identifier, + } + } else { + // The narrow index, not the wide one: only a transformation Tally was + // measured performing may settle which master was meant. Everything the + // wide fold reaches and this does not falls through to `collect_candidates` + // below, where it is offered as `NormalizedEqual` for a human to confirm. + match catalog + .by_binding_key + .get(&entity.binding_key) + .map(Vec::as_slice) + { + // §9.4d measured **ledgers**. Whether stock items match by the + // same rule is not merely unmeasured, it was never sent — so a + // folded stock-item name may suggest and may not resolve. Byte + // equality is unaffected: it needs no fold and is checked above. + Some([index]) if catalog.class == MasterClass::Ledger => BindingStatus::Bound { + catalog_name: catalog.entries[*index].name.clone(), + basis: BindingBasis::NormalizedName, + }, + // A single folded match that the class does not license is not an + // ambiguity — nothing shares its key. `NameAmbiguous` would tell a + // consumer that several masters collided when exactly one did not + // qualify, which is a different fact with a different remedy. + Some([_]) => unresolved_status( + catalog, + entity, + UnboundReason::NearMiss, + IdentifierEvidence { + exact, + matches: &identifier_matches, + withheld_holders, + }, + budget, + memo, + ), + Some(_) => unresolved_status( + catalog, + entity, + UnboundReason::NameAmbiguous, + IdentifierEvidence { + exact, + matches: &identifier_matches, + withheld_holders, + }, + budget, + memo, + ), + None => { + let (candidates, masters_found) = + remembered_candidates(catalog, entity, &identifier_matches, memo); + let reason = if !candidates.is_empty() { + UnboundReason::NearMiss + } else if masters_found > MAX_PREFIX_FAMILY { + UnboundReason::NoDiscriminatingCandidate + } else { + UnboundReason::NoCandidate + }; + unresolved_from( + catalog, + entity, + reason, + candidates, + masters_found.max(withheld_holders), + budget, + ) + } + } + }; + + EntityBinding { + position: entity.position, + source_name: entity.name.clone(), + status, + } +} + +fn unresolved_status( + catalog: &MasterCatalog, + entity: &SourceEntity, + reason: UnboundReason, + evidence: IdentifierEvidence<'_>, + budget: &mut usize, + memo: &mut SearchMemo<'_>, +) -> BindingStatus { + let IdentifierEvidence { + exact, + matches: identifier_matches, + withheld_holders, + } = evidence; + let (mut candidates, masters_found) = + remembered_candidates(catalog, entity, identifier_matches, memo); + if let Some(index) = exact { + // Byte equality, labelled as itself. Adding it as `NormalizedEqual` + // hid the strongest name evidence there is and sorted it behind the + // identifier that outranked it — on a conflict produced *because* byte + // equality was observed, which left the operator reading the two facts + // that disagreed without being told one of them was exact. + candidates.retain(|(candidate, _)| *candidate != index); + candidates.push((index, CandidateRule::ExactName)); + } + unresolved_from( + catalog, + entity, + reason, + candidates, + masters_found.max(withheld_holders), + budget, + ) +} + +fn unresolved_from( + catalog: &MasterCatalog, + entity: &SourceEntity, + reason: UnboundReason, + candidates: Vec<(usize, CandidateRule)>, + masters_found: usize, + budget: &mut usize, +) -> BindingStatus { + let mut ordered = candidates; + ordered.sort_by(|left, right| candidate_order(catalog, left, right)); + // The variant is derived here, in one place, from the same facts that chose + // the reason — so "empty" can never mean something the variant does not say. + let candidates = if ordered.is_empty() { + if masters_found > 0 { + Candidates::Withheld { + found: masters_found, + } + } else { + Candidates::None + } + } else { + let capped = ordered.len().min(MAX_CANDIDATES_PER_ENTITY); + let listed = ordered + .into_iter() + .take(MAX_CANDIDATES_PER_ENTITY) + .map_while(|(index, rule)| { + let catalog_name = &catalog.entries[index].name; + *budget = budget.checked_sub(catalog_name.len())?; + Some(Candidate { + catalog_name: catalog_name.clone(), + rule, + }) + }) + .collect::>(); + let found = masters_found.max(capped); + if listed.len() < found { + Candidates::Truncated { listed, found } + } else { + Candidates::Listed { listed } + } + }; + let unresolved = Unresolved { + reason, + unresolved_identity: entity.identifiers.clone(), + candidates, + }; + if matches!(reason, UnboundReason::NoCandidate) { + BindingStatus::Unmatched(unresolved) + } else { + BindingStatus::Ambiguous(unresolved) + } +} + +/// `collect_candidates` behind its memo, and the only way to reach it. +/// +/// The first version of this memo sat inside `unresolved_status`, which reaches +/// the search for an identifier conflict and for a name ambiguity — but **not** +/// for an ordinary near miss, which is the one case the cost was reported +/// against. Routing one more call site would have fixed that instance and left +/// the next one to be noticed; one entry point makes it structural. +fn remembered_candidates( + catalog: &MasterCatalog, + entity: &SourceEntity, + identifier_matches: &BTreeSet, + memo: &mut SearchMemo<'_>, +) -> (Vec<(usize, CandidateRule)>, usize) { + let key = (entity.key.clone(), identifier_matches.clone()); + if let Some(remembered) = memo.seen.get(&key) { + return remembered.clone(); + } + let computed = collect_candidates(catalog, entity, identifier_matches); + // Entry *count* alone does not bound a memo whose keys and values are + // themselves collections, so the key is still size-tested. The **value** is + // not, any more: `collect_candidates` now returns at most + // `MAX_CANDIDATES_PER_ENTITY` candidates, so every result is small enough + // to hold. Refusing to hold large ones read as prudence and was the + // opposite — it excluded from the memo exactly the expensive repeated + // search the memo exists for, and a name reaching twenty thousand masters + // through shared tokens re-ran it once per row. + debug_assert!(computed.0.len() <= MAX_CANDIDATES_PER_ENTITY); + let worth_holding = memo + .repeated + .contains(&(entity.key.as_str(), entity.identifiers.as_slice())) + && key.1.len() <= MAX_CANDIDATES_PER_ENTITY; + if worth_holding && memo.seen.len() < MAX_CANDIDATE_MEMO_ENTRIES { + memo.seen.insert(key, computed.clone()); + } + computed +} + +// Counts holder sets actually materialized, for the same reason as the search +// counter below: the *outcome* of expanding a family and of refusing to is +// identical — a conflict either way — so a test asserting the outcome cannot +// tell whether the work was done. +#[cfg(test)] +thread_local! { + pub(crate) static HOLDER_EXPANSIONS: std::cell::Cell = + const { std::cell::Cell::new(0) }; +} + +// Counts searches that actually ran, so a test can prove the memo is consulted +// rather than assume it. A test asserting only that the answer is right passes +// whether or not the search ran — which is exactly how the near-miss path +// stayed unmemoized through a green suite. +#[cfg(test)] +thread_local! { + pub(crate) static CANDIDATE_SEARCHES: std::cell::Cell = + const { std::cell::Cell::new(0) }; +} + +/// Produces every defensible master, each labelled with the rule that surfaced +/// it. The strongest rule wins where several apply. Nothing here ranks by +/// similarity, and nothing here chooses. +fn collect_candidates( + catalog: &MasterCatalog, + entity: &SourceEntity, + identifier_matches: &BTreeSet, +) -> (Vec<(usize, CandidateRule)>, usize) { + #[cfg(test)] + CANDIDATE_SEARCHES.with(|count| count.set(count.get() + 1)); + // Masters this name reaches by prefix. The key index is ordered, so this is + // a range walk rather than a scan of the catalog per entity. + let extending = if entity.key.chars().count() >= MIN_PREFIX_KEY_CHARS { + catalog + .by_key + .range(entity.key.clone()..) + .take_while(|(key, _)| key.starts_with(&entity.key)) + .filter(|(key, _)| *key != &entity.key) + .flat_map(|(_, holders)| holders.iter().copied()) + .collect::>() + } else { + BTreeSet::new() + }; + // Beyond the bound they are a family this name does not separate, and an + // arbitrary capped slice of one omitted the right master about a third of + // the time against live books — 65.6% present, against 100% once families + // over the bound were withheld. Counted, and withheld rather than listed. + // `TALLY_PROTOCOL_REFERENCE.md` §9.4c states the rule; `TEST_CORPUS.md` + // §9.1 carries the counts, the cause and what they do not cover. + let withheld = if extending.len() > MAX_PREFIX_FAMILY { + extending.clone() + } else { + BTreeSet::new() + }; + + let mut best: BTreeMap = BTreeMap::new(); + let mut offer = |index: usize, rule: CandidateRule| { + best.entry(index) + .and_modify(|current| { + if rule.rank() < current.rank() { + *current = rule; + } + }) + .or_insert(rule); + }; + + // A decisive rule reaches a master on its own evidence, so it still applies + // to a member of a withheld family: the identifier, or the whole key, is + // exactly what separates that one from its siblings. + for index in identifier_matches { + offer(*index, CandidateRule::SharedIdentifier); + } + for index in catalog.by_key.get(&entity.key).into_iter().flatten() { + offer(*index, CandidateRule::NormalizedEqual); + } + // The masters that *caused* a name ambiguity are the ones the resolving + // fold collided, and they are not always reachable from the wide key: the + // wide fold replaces `-` but not `/`, so `AB/CD` and `AB CD` are one master + // to `verified_fold` and two to `master_identity_key`. Listing only the + // wide key's holders reported an ambiguity with a complete-looking list of + // one, omitting the master the operator was being asked to choose between. + // + // Taking the holders from the index the reason was decided on is right + // regardless of how the two folds relate. Widening `master_identity_key` to + // fold `/` as well would fix this one example, change prefix families, + // token sets and memo keys across the whole module on the strength of it, + // and still leave the candidate list assembled from a key that is not the + // one the ambiguity was found in. + for index in catalog + .by_binding_key + .get(&entity.binding_key) + .into_iter() + .flatten() + { + offer(*index, CandidateRule::NormalizedEqual); + } + if withheld.is_empty() { + for index in &extending { + offer(*index, CandidateRule::CatalogPrefix); + } + } + + // Weaker rules must not reinstate what the prefix pass withheld. A token + // shared across a family *is* the family, and re-listing 25 of them is the + // arbitrary slice the withholding exists to prevent — reachable whenever + // the family stays under the common-token threshold, as 30 rows in a + // 330-master catalog do. + if !entity.key.is_empty() { + // One pass over character boundaries; recomputing a prefix length per + // split made this quadratic in a field the source parser admits at 4 KiB. + for (characters, (split, _)) in entity.key.char_indices().enumerate() { + if characters < MIN_PREFIX_KEY_CHARS { + continue; + } + for index in catalog + .by_key + .get(&entity.key[..split]) + .into_iter() + .flatten() + .filter(|index| !withheld.contains(index)) + { + offer(*index, CandidateRule::SourcePrefix); + } + } + } + for token in tokens_of(&entity.key) { + if catalog.common_tokens.contains(&token) { + continue; + } + for index in catalog + .by_token + .get(&token) + .into_iter() + .flatten() + .filter(|index| !withheld.contains(index)) + { + offer(*index, CandidateRule::SharedToken); + } + } + + // The total is the union: a withheld family and the candidates still worth + // listing are not necessarily the same masters, so the larger of the two + // counts would under-report what the name reaches. + let found = best + .keys() + .copied() + .chain(withheld) + .collect::>() + .len(); + // Indices, not names — cloning every match before the cap and the budget + // discarded most of the work, once per entry of an admitted draft. + let mut listed = best.into_iter().collect::>(); + // Capped **here**, in the order the caller sorts into, rather than + // downstream after the whole union has been returned. Two reasons, and the + // second is the one that was biting: + // + // 1. The return value was unbounded. A name reaching a large family through + // shared tokens built a vector of that whole family per source row, of + // which at most `MAX_CANDIDATES_PER_ENTITY` survive. + // 2. The memo refused to hold a result larger than that cap, so exactly the + // expensive repeated search was the one never remembered, and a draft + // repeating that name re-ran it once per row. Bounding the result makes + // every result cacheable, which is what the memo was for. + // + // `found` is computed above from the full union, so the count an operator + // sees is unaffected by the cap; only the listing is. + listed.sort_by(|left, right| candidate_order(catalog, left, right)); + listed.truncate(MAX_CANDIDATES_PER_ENTITY); + (listed, found) +} + +/// How candidates are ordered wherever they are ordered: by the rule that +/// reached them, then by the master's name. +/// +/// Defined once because `collect_candidates` truncates in this order and +/// `unresolved_from` sorts in it, and a disagreement between the two would +/// silently drop a candidate that should have been listed. +fn candidate_order( + catalog: &MasterCatalog, + left: &(usize, CandidateRule), + right: &(usize, CandidateRule), +) -> std::cmp::Ordering { + left.1.rank().cmp(&right.1.rank()).then_with(|| { + catalog.entries[left.0] + .name + .cmp(&catalog.entries[right.0].name) + }) +} + +/// A name is retained **verbatim**, on both sides. +/// +/// An observed master name is written back to Tally byte for byte by a caller +/// that acts on a binding, so trimming it would report a spelling the book does +/// not contain. A requested source name is what byte equality is judged +/// against, so trimming it would let `Bank ` claim an exact match on `Bank` +/// while the import file still carries the trailing space. The comparison key +/// collapses surrounding whitespace anyway, so the two still meet as a +/// normalized match — which is a bind the write gate does not admit, and that +/// is the correct, loud outcome. +fn validated_name(value: &str) -> Result { + validate_name_bounds(value)?; + Ok(value.to_string()) +} + +fn validate_name_bounds(value: &str) -> Result<(), MasterBindingError> { + if value.trim().is_empty() { + return Err(MasterBindingError::NameBlank); + } + if value.chars().any(char::is_control) { + return Err(MasterBindingError::NameUnsafe); + } + if value.chars().count() > MAX_NAME_CHARS { + return Err(MasterBindingError::NameTooLong); + } + Ok(()) +} + +/// Folds the punctuation an operator happened to type: NFC-equivalent dash and +/// quote variants become ASCII, case is lowered, whitespace runs collapse. +/// Nothing else is folded — no stemming, no transliteration, no vowel removal. +/// +/// **This is a contract, not an implementation detail.** Anything in this crate +/// that decides whether two operator-typed strings are "the same" — master +/// names here, and voucher numbers or voucher-type names elsewhere — must fold +/// through this one function. A second, subtly different normaliser is exactly +/// the divergence ADR 0016 exists to end, and it would diverge silently: +/// the two agree on every name anyone tests by hand and disagree on the +/// punctuation nobody thinks to try. +/// +/// The corollary is that changing what this folds changes every consumer's +/// notion of sameness at once. Widen it only with the same care as a wire +/// format, and never to make one caller's case pass. +pub(crate) fn comparison_key(value: &str) -> String { + value + .nfc() + .flat_map(|character| match character { + '\u{2010}' | '\u{2011}' | '\u{2012}' | '\u{2013}' | '\u{2014}' | '\u{2015}' + | '\u{2212}' => vec!['-'], + '\u{2018}' | '\u{2019}' | '\u{201a}' | '\u{201b}' => vec!['\''], + '\u{201c}' | '\u{201d}' | '\u{201e}' | '\u{201f}' => vec!['"'], + other => other.to_lowercase().collect(), + }) + .collect::() + .split_whitespace() + .collect::>() + .join(" ") +} + +/// The **wide** fold: which masters are worth showing a human. +/// +/// This is deliberately looser than anything measured, and it may never decide +/// a binding. `verified_fold` does that. The separation is the whole design: +/// §9.4b verified three transformations and marks the rest UNVERIFIED, and its +/// own remedy is that a looser fold may *suggest* while only the measured ones +/// resolve. So the reverse hyphen direction, collapsed whitespace runs, leading +/// whitespace and the Unicode dash variants all live here, where the worst they +/// can do is put the right master in front of an operator. +/// +/// An earlier version of this module let this fold bind. It read naturally and +/// was wrong: `X - Y` is a common ledger convention — six of seventeen +/// hyphenated names in the observed books take that shape — and reaching it +/// from `X Y` needs the measured hyphen step **and** a whitespace run collapsed, +/// which nothing measured. Answering from here was a Bridge guess wearing +/// Tally's authority, and `TEST_CORPUS.md` §9 caught it binding that way +/// against a live instance. +/// +/// It is a **separate** function rather than a widening of `comparison_key` +/// precisely because that one is shared: voucher numbers and voucher-type names +/// fold through it too, and §9.4b says nothing about those. One fold per notion +/// of sameness, each named for the question it answers — and here there are two +/// notions, because "could this be the master?" and "is this the master?" are +/// different questions with different evidence behind them. +fn master_identity_key(value: &str) -> String { + comparison_key(value) + .replace('-', " ") + .split_whitespace() + .collect::>() + .join(" ") +} + +/// The fold that may **resolve** a name to a master: exactly the equivalences +/// `TALLY_PROTOCOL_REFERENCE.md` §9.4d measured on the SKU this writes to. +/// +/// §9.4b measured Edit Log 7.0 Educational and marked most of this UNVERIFIED, +/// so an earlier version of this module resolved on three transformations only +/// and offered the rest as candidates. §9.4d re-ran that measurement on +/// **licensed TallyPrime 7.1**, read the day book back to see which master each +/// name actually reached, and found the gateway wider than the Educational +/// scope allowed anyone to claim: +/// +/// - ASCII case folds; +/// - leading and trailing whitespace is ignored; +/// - an internal run of spaces collapses; +/// - **space, `-` and `/` are one separator**, in both directions. +/// +/// Everything else is exact on codepoints. So the two rules that matter are +/// both negative, and neither is guessable from appearance: +/// +/// **An en dash and an underscore are not separators.** They were sent and +/// rejected. A fold that treats "punctuation" or "separators" as a class is +/// wider than the gateway and merges masters Tally keeps apart — which is why +/// the separator set here is written out rather than described. +/// +/// **Canonical equivalence is not folded.** An NFD spelling of an NFC master +/// was rejected here too, consistent with the exact-codepoint finding recorded +/// against this same release. Normalizing before comparing would resolve a name +/// onto a master the gateway keeps apart. It reads like decoding rather than +/// folding, which is how it survived two audits of this function. +/// +/// Both hyphen directions are measured now, so this is symmetric and one key +/// per side is enough — the asymmetric index an earlier version needed is gone. +fn verified_fold(value: &str) -> String { + value + .chars() + .map(|character| match character { + '-' | '/' => ' ', + other => other.to_ascii_lowercase(), + }) + .collect::() + // ASCII space only. A tab and a no-break space were never sent, so they + // stay ordinary characters rather than joining the separator set on the + // strength of looking like whitespace. + .split(' ') + .filter(|part| !part.is_empty()) + .collect::>() + .join(" ") +} + +/// Splits a comparison key into words. +/// +/// The separator set is defined **positively** — whitespace, and ASCII +/// punctuation — rather than as "not alphanumeric". The difference is not +/// pedantic: `char::is_alphanumeric` is **false for a Devanagari virama**, the +/// halant that joins consonants, and false for a nukta. Splitting on it tore +/// Indic ledger names apart at the joins, so `राय एण्ड सन्स` yielded one usable +/// token instead of three and `ट्रेडर्स` was cut to a fragment. Those names are +/// in the books this binder reads. +/// +/// So: an ASCII class may decide a question about ASCII characters, and +/// everything outside ASCII — letters, digits, marks, joiners alike — is word +/// content. That keeps the rule correct for scripts nobody here has tested, +/// which is the property worth having. +fn tokens_of(key: &str) -> BTreeSet { + key.split(|character: char| { + character.is_whitespace() || (character.is_ascii() && !character.is_ascii_alphanumeric()) + }) + .filter(|token| token.chars().count() >= MIN_TOKEN_CHARS) + .map(str::to_string) + .collect() +} + +/// Extracts every identifier a name carries, in canonical form. +/// +/// A numeric run may hold `-` and `/` internally, so a punctuated account +/// number and a plain one agree; it may not hold spaces, so separated digit +/// groups fail closed to a near-miss rather than fusing into a false +/// identifier. Digits that sit inside a mixed letter-and-digit token belong to +/// that token's code and are never also emitted on their own — otherwise +/// `Part AB12345678` would collide with an unrelated `Bank 12345678`. +/// +/// Refuses rather than truncates when a name carries more identifiers than the +/// bound: silently keeping the first few can turn a conflict into a bind by +/// discarding the identifier that pointed elsewhere. +fn extract_identifiers(value: &str) -> Result, MasterBindingError> { + let mut identifiers = BTreeSet::new(); + // A mask and the digits it hides are often written apart — `**** 12345678` + // is the same statement as `********12345678`, and reading tokens + // independently lost the relationship between them. + let mut previous_was_mask = false; + for token in value.split(char::is_whitespace) { + if token.is_empty() { + continue; + } + let masked_here = is_mask_punctuated(token) || is_mask_alphabetic(token); + let masked = masked_here || previous_was_mask; + // A token carrying no alphanumeric content is a delimiter, not a value, + // and a delimiter between a mask and its suffix does not unmask it: + // `XXXX - 12345678` says exactly what `XXXX 12345678` says. Clearing + // the state here let a `-` or a `/` walk the suffix out as a whole + // account number. + if masked_here { + previous_was_mask = true; + } else if token.chars().any(char::is_alphanumeric) { + previous_was_mask = false; + } + // Only the separators §9.4d measured may be discarded. Filtering to + // alphanumerics dropped **every** ASCII punctuation mark, so + // `AB_123456` and `AB-123456` canonicalized alike and one identifier + // bound the other's master — while §9.4d had sent an underscore and + // watched Tally *reject* it. The evidence for this fold is one + // measurement about hyphens and slashes; everything else stays content. + let canonical = token + .chars() + .filter(|character| !matches!(character, '-' | '/')) + .map(|character| character.to_ascii_uppercase()) + .collect::(); + let digits = canonical.chars().filter(char::is_ascii_digit).count(); + let letters = canonical.chars().filter(char::is_ascii_alphabetic).count(); + // Canonicalization keeps ASCII, and so does the digit-run split below. + // Everything else in a token is silently discarded, and what survives + // is a code or a number the name never contained: a Devanagari party + // name fused to `AB12345678` yielded `AB12345678`, and `12345678` + // followed by Devanagari digits yielded `12345678` — each reaching an + // unrelated master that the same shape spelled in ASCII never would. + // + // Guarding "non-ASCII letters" was the first attempt and was too + // narrow: `char::is_alphabetic` is false for a Devanagari digit, so the + // numerals walked straight through it. The admitted set is positive + // instead — ASCII, plus the dash variants this module already treats as + // separators — because the question is not which scripts exist but + // which characters canonicalization is entitled to drop. + let foreign_content = token + .chars() + .any(|character| !character.is_ascii() && !DASH_VARIANTS.contains(&character)); + // The threshold counts the characters that *identify*, not the bytes + // that spell them. `len()` is UTF-8 bytes, and an admitted dash variant + // is three of them, so `AB\u{2013}123` measured 8 and cleared a bound meant + // for eight characters while carrying five alphanumerics — decisive on + // the strength of one punctuation mark. Its ASCII twin `AB-123` reduces + // to `AB123` and is refused, so the same code bound or did not + // depending on which dash the document happened to use. Counting + // letters and digits also closes the padding shape the byte count never + // saw: `AB\u{2013}\u{2013}\u{2013}123` is still five alphanumerics. + let identifying = digits + letters; + if identifying >= MIN_CODE_IDENTIFIER_CHARS + && digits >= MIN_CODE_IDENTIFIER_DIGITS + && letters >= 2 + && !foreign_content + // A separator works in both directions, so the date guard has to + // run on both spellings. Removing `-` can *reveal* a date — + // `2025-09-11` becomes `20250911` — and it can just as easily + // *hide* two: `DATED20250911-20250912` fuses into one sixteen-digit + // run that reads as no date at all, and `is_period` does not see it + // either, because its eight-digit case admits a year followed by a + // year and `0911` is neither. A date range then identified, and a + // period label is the one thing two unrelated masters most reliably + // share. + && !carries_plausible_date(&canonical) + && !carries_plausible_date(token) + && !masked + && !is_period(token) + && !is_masked(&canonical) + // The upper bound stays on the canonical's **bytes**, because it + // bounds what this index stores and clones rather than what + // identifies. A byte cap admits no more characters than it says, + // so it is the conservative half of the pair. + && canonical.len() <= MAX_IDENTIFIER_CHARS + { + identifiers.insert(Identifier { + kind: IdentifierKind::Code, + value: canonical, + }); + } + // Digits sitting beside letters belong to that token, whether or not it + // qualified as a code. Emitting them separately let `Part A12345678` + // reach an unrelated `Bank 12345678` through the one-letter gap that + // the code test rejects — a token either identifies by its whole shape + // or not at all. + // + // The test is **Unicode alphabetic**, not ASCII. The books observed + // here carry Devanagari, Tamil and Bengali ledger names, and an + // ASCII-only guard read `पार्टी12345678` as digits standing alone, + // binding a party to an unrelated `Bank 12345678`. + if token.chars().any(char::is_alphabetic) || masked || foreign_content { + continue; + } + for run in token.split(|character: char| { + !(character.is_ascii_digit() || character == '-' || character == '/') + }) { + let digits = run.chars().filter(char::is_ascii_digit).collect::(); + // Each side of a punctuated run is tested before the separator is + // removed. `20250911-20250912` fuses to sixteen digits, which is no + // length `is_plausible_date` recognizes, and `is_period` reads + // neither half as a year range — so a date *range* walked through a + // guard that a single date does not. Checking the components is + // where the check belonged: the fusing is what hid them. + let component_is_a_date = run.split(DASH_VARIANTS).any(is_plausible_date); + if digits.len() >= MIN_NUMERIC_IDENTIFIER_DIGITS + && digits.len() <= MAX_IDENTIFIER_CHARS + && !component_is_a_date + && !is_plausible_date(&digits) + && !is_period(run) + { + identifiers.insert(Identifier { + kind: IdentifierKind::Numeric, + value: digits, + }); + } + } + } + if identifiers.len() > MAX_IDENTIFIERS_PER_NAME { + return Err(MasterBindingError::TooManyIdentifiers); + } + Ok(identifiers.into_iter().collect()) +} + +/// A value written with mask punctuation is partial by construction, so the +/// digits it does expose are a suffix rather than the number. +/// +/// `********12345678` split cleanly on the asterisks and yielded its visible +/// eight digits as though they were the whole account. `is_masked` guards the +/// code branch only, because a mask spelled with letters is caught by the +/// letter test; a mask spelled with punctuation reaches the numeric branch, +/// where every non-digit is an ordinary delimiter. +/// A mask spelled with letters hides its digits exactly as one spelled with +/// punctuation does. `is_masked` already reads the shape inside a single token, +/// so `XXXX1234` never became a code; written a space apart as `XXXX 12345678` +/// the same statement lost its mask, and the visible suffix escaped as though +/// it were a whole account number — enough to bind a missing purchase ledger to +/// a sole live sales one carrying the same last eight. +/// +/// Two characters minimum, and letters only: a lone `A` is an ordinary name +/// word, and a token carrying digits is already judged whole by `is_masked`. +fn is_mask_alphabetic(token: &str) -> bool { + let canonical = token + .chars() + .filter(|character| character.is_ascii_alphanumeric()) + .map(|character| character.to_ascii_uppercase()) + .collect::(); + canonical.len() >= 2 + && canonical + .chars() + .all(|character| character.is_ascii_alphabetic()) + && is_masked(&canonical) +} + +fn is_mask_punctuated(token: &str) -> bool { + if token + .chars() + .any(|character| matches!(character, '*' | '#' | '\u{2022}' | '\u{00d7}')) + { + return true; + } + // Those four glyphs say "masked" on their own. Everything else has to be + // recognized by **shape**, because enumerating them lost to `........1234` + // and `____ 1234` and would have lost to the next spelling too. A run of + // one repeated non-alphanumeric is a mask; ordinary punctuation does not + // repeat that way, so `S.K. Traders` and `(5550001001)` are untouched. + let mut run: Option<(char, usize)> = None; + for character in token.chars() { + if character.is_alphanumeric() { + run = None; + continue; + } + run = match run { + Some((previous, count)) if previous == character => Some((character, count + 1)), + _ => Some((character, 1)), + }; + if run.is_some_and(|(_, count)| count >= MIN_MASK_RUN) { + return true; + } + } + false +} + +/// Three, because two is a legitimate ellipsis of a kind operators do write and +/// a doubled separator is ordinary noise. Erring long here only costs a bind. +const MIN_MASK_RUN: usize = 3; + +/// The separators an operator writes a range with. The comparison key already +/// folds these dash variants to ASCII; the period boundary has to admit the +/// same set, or `FY2025\u{2013}26` fuses where `FY2025-26` splits. +const DASH_VARIANTS: [char; 9] = [ + '-', '/', '\u{2010}', '\u{2011}', '\u{2012}', '\u{2013}', '\u{2014}', '\u{2015}', '\u{2212}', +]; + +/// A masked value exposes a non-unique suffix and identifies nothing. +/// +/// `XXXXX1234X` clears every length and composition test — ten characters, six +/// letters, four digits, no period — while the only information in it is a last +/// four that any number of parties share. Two unrelated ledgers carrying the +/// same mask would bind to each other. +/// +/// Recognized by its letters being a single repeated character, which is what a +/// mask is and what an identity-bearing code never is: `PH01AB00` and a +/// registration number both carry distinct letters. +fn is_masked(canonical: &str) -> bool { + let mut letters = canonical.chars().filter(char::is_ascii_alphabetic); + match letters.next() { + Some(first) => letters.all(|letter| letter == first), + None => false, + } +} + +/// A period label identifies a period, not a party or an item. Two unrelated +/// ledgers routinely share one, and identifier-first matching would bind the +/// source to whichever exists before it ever compared the names. +/// +/// Applied to the **raw token**, because operators write ranges with the very +/// separators canonicalization strips: `FY2025-26` fuses to `FY202526`, whose +/// six-digit run reads as no period at all, and the label walked straight into +/// being a code. Splitting on the separator first keeps `FY2025` and `26` +/// legible as what they are. +/// +/// The test is on the **numbers**, not the words: every part carries only +/// alphabetic markers and numbers that read as a year or a small ordinal, and +/// at least one number appears. Capping the length of the alphabetic run was an +/// earlier attempt that kept losing to longer spellings — a month name can be +/// any length; a year cannot. +/// +/// An identity-bearing code survives: `PH-01A-B00` splits to a `PH` carrying no +/// number at all, and `AB12345678` holds a run no calendar would produce. Like +/// every exclusion here it can only make a bind *less* likely. +fn is_period(token: &str) -> bool { + let mut any_number = false; + for part in token.split(DASH_VARIANTS) { + let canonical = part + .chars() + .filter(char::is_ascii_alphanumeric) + .map(|character| character.to_ascii_uppercase()) + .collect::(); + if canonical.is_empty() || !part_reads_as_period(&canonical, &mut any_number) { + return false; + } + } + any_number +} + +fn part_reads_as_period(canonical: &str, any_number: &mut bool) -> bool { + let mut rest = canonical; + while !rest.is_empty() { + let alphabetic = rest.starts_with(|character: char| character.is_ascii_alphabetic()); + let split = rest + .find(|character: char| character.is_ascii_alphabetic() != alphabetic) + .unwrap_or(rest.len()); + let (run, tail) = rest.split_at(split); + rest = tail; + if alphabetic { + continue; + } + let value = run.parse::().unwrap_or(u32::MAX); + let reads_as_period = match run.len() { + 1 | 2 => (1..=99).contains(&value), + 4 => (1900..=2199).contains(&value), + // A range written without its separator. `FY2025-26` splits and is + // read part by part; `FY202425` arrives whole, and every length + // test above missed it — so the token passed as a code and bound a + // missing `Purchases FY202425` to a sole live `Sales FY202425`. + // Both spellings of the suffix occur: `202425` and `20242025`. + 6 | 8 => { + let (lead, suffix) = run.split_at(4); + let lead = lead.parse::().unwrap_or(u32::MAX); + let suffix_value = suffix.parse::().unwrap_or(u32::MAX); + (1900..=2199).contains(&lead) + && if suffix.len() == 2 { + (1..=99).contains(&suffix_value) + } else { + (1900..=2199).contains(&suffix_value) + } + } + _ => false, + }; + if !reads_as_period { + return false; + } + *any_number = true; + } + true +} + +/// Whether any maximal digit run inside a token reads as a date. +/// +/// `is_plausible_date` guarded the numeric branch only, so `DATED20250911` +/// reached the code branch untouched: `is_period` does not see it either, +/// because its eight-digit case admits a year followed by a year and `0911` is +/// neither. Two unrelated ledgers sharing a fused date label then bound to each +/// other on it. +/// +/// **This is spelling-specific, deliberately, and it is not the general rule.** +/// It is called on both the raw token and its canonical form because removing a +/// separator can reveal a date and can equally hide two, and those are two +/// different maximal-run decompositions. A partly separated range such as +/// `2025-0911-2025-0912` still escapes all three date guards, and the obvious +/// generalization — scanning every eight-digit *window* of the token's +/// concatenated digits — was considered and rejected: a sixteen-digit account +/// number containing a date-shaped window would then be refused, and that is a +/// strong identifier being thrown away to catch a label. Erring toward refusal +/// is right when the alternative is a wrong-party bind; it is not right when it +/// costs the identifiers this module exists to use. If a better rule is found, +/// it belongs here, replacing all three. +fn carries_plausible_date(canonical: &str) -> bool { + canonical + .split(|character: char| !character.is_ascii_digit()) + .any(is_plausible_date) +} + +/// An eight-digit run that reads as a calendar date in any order this project +/// admits is a date, not an identifier. Recognizing only `YYYYMMDD` left +/// `01012026` binding a source to an unrelated master that shares its period +/// label. Being generous here can only make a bind *less* likely, which is the +/// safe direction for a rule whose failure mode is money against the wrong +/// party. +fn is_plausible_date(digits: &str) -> bool { + if digits.len() != 8 { + return false; + } + let number = |range: std::ops::Range| digits[range].parse::().unwrap_or(0); + let (first, second, third, fourth) = (number(0..4), number(4..6), number(6..8), number(4..8)); + let (day, month) = (number(0..2), number(2..4)); + let year_first = + (1900..=2199).contains(&first) && (1..=12).contains(&second) && (1..=31).contains(&third); + // DDMMYYYY and MMDDYYYY are indistinguishable from each other without a + // locale, so either reading is enough to disqualify the run. + let year_last = (1900..=2199).contains(&fourth) + && ((1..=31).contains(&day) && (1..=12).contains(&month) + || (1..=12).contains(&day) && (1..=31).contains(&month)); + year_first || year_last +} + +#[cfg(test)] +#[path = "master_binding_tests.rs"] +mod tests; diff --git a/src-tauri/crates/bridge-tally-core/src/master_binding_tests.rs b/src-tauri/crates/bridge-tally-core/src/master_binding_tests.rs new file mode 100644 index 000000000..095d2edcf --- /dev/null +++ b/src-tauri/crates/bridge-tally-core/src/master_binding_tests.rs @@ -0,0 +1,2606 @@ +//! Every name, code and number here is fabricated from a placeholder +//! alphabet — Greek-letter party names, `PH-` stock codes, and numbers drawn +//! from the `55500000xx` placeholder block. Nothing is edited down from an +//! observed book, and none of it is evidence about any Tally instance: these +//! tests establish the behaviour of the binding rules only. + +use super::*; + +fn ledgers(names: &[&str]) -> MasterCatalog { + MasterCatalog::new(MasterClass::Ledger, names).expect("fabricated catalog is valid") +} + +fn entity(name: &str) -> SourceEntity { + SourceEntity::new(1, name).expect("fabricated source name is valid") +} + +fn bound(catalog: &MasterCatalog, entities: &[SourceEntity]) -> BindingReport { + bind(catalog, entities).expect("fabricated entity list is within bounds") +} + +fn bind_one_name(catalog: &MasterCatalog, name: &str) -> EntityBinding { + bound(catalog, &[entity(name)]) + .entities() + .first() + .cloned() + .expect("one entity in, one binding out") +} + +fn candidate_names(binding: &EntityBinding) -> Vec<&str> { + binding + .unresolved() + .expect("binding did not resolve") + .candidates + .listed() + .iter() + .map(|candidate| candidate.catalog_name.as_str()) + .collect() +} + +fn reason(binding: &EntityBinding) -> UnboundReason { + binding + .unresolved() + .expect("binding did not resolve") + .reason +} + +// --- inputs are valid by construction ------------------------------------- + +#[test] +fn an_unread_book_is_an_error_not_an_empty_report() { + // Binding against a book nobody read is the failure this contract exists + // to prevent, so it cannot be expressed as "everything is missing". + let empty: [&str; 0] = []; + assert_eq!( + MasterCatalog::new(MasterClass::Ledger, empty), + Err(MasterBindingError::CatalogEmpty) + ); +} + +#[test] +fn a_duplicate_master_name_refuses_the_catalog() { + assert_eq!( + MasterCatalog::new(MasterClass::Ledger, ["Alpha Traders", "Alpha Traders"]), + Err(MasterBindingError::CatalogDuplicateName) + ); +} + +#[test] +fn unusable_names_are_refused_at_the_boundary() { + assert_eq!( + MasterCatalog::new(MasterClass::Ledger, [" "]), + Err(MasterBindingError::NameBlank) + ); + assert_eq!( + MasterCatalog::new(MasterClass::Ledger, ["Alpha\u{7}Traders"]), + Err(MasterBindingError::NameUnsafe) + ); + let long = "A".repeat(MAX_NAME_CHARS + 1); + assert_eq!( + MasterCatalog::new(MasterClass::Ledger, [long.as_str()]), + Err(MasterBindingError::NameTooLong) + ); + assert_eq!(SourceEntity::new(0, ""), Err(MasterBindingError::NameBlank)); +} + +#[test] +fn the_measured_transformations_compose() { + // Twelve single-axis results license each transformation alone and say + // nothing about applying several at once — which is what a canonical form + // does on every comparison. Two reviewers raised that independently, so + // §9.4d measured it rather than arguing it: eight composed variants, all + // matched, all confirmed by day-book readback against the intended master. + let catalog = ledgers(&[ + "MB PILOT ALPHA (5550001001)", + "MB-PROBE-LEDGER-A", + "Beta Supply", + ]); + for (supplied, expected) in [ + ( + " mb pilot alpha (5550001001) ", + "MB PILOT ALPHA (5550001001)", + ), + ("mb-pilot-alpha-(5550001001)", "MB PILOT ALPHA (5550001001)"), + ( + " mb-pilot-alpha-(5550001001) ", + "MB PILOT ALPHA (5550001001)", + ), + ( + "MB/PILOT ALPHA (5550001001)", + "MB PILOT ALPHA (5550001001)", + ), + ("mb-pilot alpha/(5550001001)", "MB PILOT ALPHA (5550001001)"), + ( + " mb-pilot/alpha (5550001001) ", + "MB PILOT ALPHA (5550001001)", + ), + (" mb probe ledger a ", "MB-PROBE-LEDGER-A"), + ] { + assert_eq!( + bind_one_name(&catalog, supplied).bound_name(), + Some(expected), + "{supplied:?} did not compose to {expected:?}" + ); + } + + // Composition does not create equivalences out of unmeasured parts: an en + // dash stays rejected however much measured folding surrounds it. + let dashed = ledgers(&["Alpha \u{2013} Traders", "Beta Supply"]); + assert_eq!( + bind_one_name(&dashed, " alpha traders ").bound_name(), + None, + "an unmeasured transformation was carried in by composition" + ); +} + +#[test] +fn surrounding_and_repeated_whitespace_is_folded_on_both_sides() { + // §9.4d: leading whitespace, one trailing space and a collapsed internal + // run all matched on licensed 7.1, in both directions. + for (master, source) in [ + ("Alpha Traders", "Alpha Traders "), + ("Alpha Traders ", "Alpha Traders"), + ("Alpha Traders", " Alpha Traders"), + (" Alpha Traders", "Alpha Traders"), + ("Alpha Traders", "Alpha Traders"), + ("Alpha Traders", "Alpha Traders"), + ] { + let catalog = ledgers(&[master, "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, source).bound_name(), + Some(master), + "{source:?} did not reach {master:?}" + ); + } + + // An observed name is still retained byte for byte: a caller writes it back. + let catalog = ledgers(&["Alpha Traders ", "Beta Supply"]); + assert_eq!(catalog.names().next(), Some("Alpha Traders ")); + let binding = bind_one_name(&catalog, "Alpha Traders"); + assert_eq!(binding.bound_name(), Some("Alpha Traders ")); + assert_eq!( + binding.source_name, "Alpha Traders", + "a source name is recorded as the document wrote it" + ); + + // ASCII space only. A no-break space was never sent, so it stays an + // ordinary character rather than joining the separator set on the strength + // of looking like whitespace. + let nbsp = ledgers(&["Alpha\u{a0}Traders", "Beta Supply"]); + assert_eq!(bind_one_name( , "Alpha Traders").bound_name(), None); +} + +#[test] +fn near_identical_masters_are_an_ambiguity_where_they_collide_and_never_a_refused_catalog() { + // A catalog holding two names one fold or another could merge must not fail + // the whole read. Whether they are *ambiguous* is a separate question, and + // the answer changed when the fold was held to what §9.4b measured. + + // Case-only siblings do collide: ASCII case folding is verified, so both + // answer to one key and a third spelling resolves to neither. + let cased = ledgers(&["Alpha Traders", "alpha traders"]); + assert_eq!( + bind_one_name(&cased, "Alpha Traders").bound_name(), + Some("Alpha Traders"), + "byte equality still picks the exact one" + ); + let other = bind_one_name(&cased, "ALPHA TRADERS"); + assert_eq!(reason(&other), UnboundReason::NameAmbiguous); + assert_eq!(candidate_names(&other), ["Alpha Traders", "alpha traders"]); + + // Two masters differing only in trailing whitespace collapse under the + // measured fold too, so they are an ambiguity rather than a refused + // catalog — and byte equality still picks one where the source has it. + let spaced = ledgers(&["Alpha Traders", "Alpha Traders "]); + assert_eq!( + bind_one_name(&spaced, "Alpha Traders ").bound_name(), + Some("Alpha Traders "), + "byte equality outranks the fold" + ); + let ambiguous = bind_one_name(&spaced, "alpha traders"); + assert_eq!(reason(&ambiguous), UnboundReason::NameAmbiguous); + assert_eq!( + candidate_names(&ambiguous), + ["Alpha Traders", "Alpha Traders "] + ); +} + +#[test] +fn an_identifier_hint_that_yields_nothing_is_refused_rather_than_ignored() { + assert_eq!( + SourceEntity::with_identifier_hints(0, "Alpha Traders", ["not-an-identifier"]), + Err(MasterBindingError::IdentifierHintUnusable) + ); + let entity = SourceEntity::with_identifier_hints(0, "Alpha Traders", ["5550000001"]) + .expect("a usable hint is accepted"); + assert_eq!( + entity.identifiers(), + [Identifier { + kind: IdentifierKind::Numeric, + value: "5550000001".to_string(), + }] + ); +} + +#[test] +fn the_source_entity_bound_is_enforced_where_a_document_is_unbounded() { + let catalog = ledgers(&["Alpha Traders"]); + let entities = (0..=MAX_SOURCE_ENTITIES) + .map(|position| SourceEntity::new(position, "Alpha Traders").expect("valid")) + .collect::>(); + assert_eq!( + bind(&catalog, &entities), + Err(MasterBindingError::TooManySourceEntities) + ); +} + +// --- what binds ------------------------------------------------------------ + +#[test] +fn an_exact_name_binds() { + let catalog = ledgers(&["Alpha Traders", "Beta Supply"]); + let binding = bind_one_name(&catalog, "Alpha Traders"); + assert_eq!( + binding.status, + BindingStatus::Bound { + catalog_name: "Alpha Traders".to_string(), + basis: BindingBasis::ExactName, + } + ); +} + +#[test] +fn case_binds_but_an_unverified_fold_only_suggests() { + // ASCII case folding is measured, so it resolves. + let cased = ledgers(&["Alpha Traders", "Beta Supply"]); + assert_eq!( + bind_one_name(&cased, "ALPHA traders").status, + BindingStatus::Bound { + catalog_name: "Alpha Traders".to_string(), + basis: BindingBasis::NormalizedName, + } + ); + + // An en dash, a collapsed whitespace run and leading whitespace are all on + // §9.4b's unverified list. The wide fold still reaches the master, so it is + // offered — a candidate a human confirms, which is exactly what §9.4b says + // a looser fold is for. Nothing is lost here except the automatic answer. + let catalog = ledgers(&["Alpha \u{2013} Traders", "Beta Supply"]); + let binding = bind_one_name(&catalog, " alpha - TRADERS "); + assert_eq!(binding.bound_name(), None); + assert_eq!(reason(&binding), UnboundReason::NearMiss); + assert_eq!( + binding.unresolved().expect("unbound").candidates.listed(), + [Candidate { + catalog_name: "Alpha \u{2013} Traders".to_string(), + rule: CandidateRule::NormalizedEqual, + }] + ); +} + +#[test] +fn an_embedded_identifier_beats_three_wrong_name_candidates() { + // The engagement case, fabricated: the source names the party one way, the + // ledger another, and the only thing that agrees is the number the + // operator buried in the ledger name. Name matching offers three wrong + // people; the identifier decides. + let catalog = ledgers(&[ + "GAMMA (5550000001)", + "GAMMA ALPHA", + "GAMMA BETA", + "GAMMA DELTA", + ]); + let source = + SourceEntity::with_identifier_hints(3, "GAMMA. EPSILON", ["5550000001"]).expect("valid"); + let report = bound(&catalog, &[source]); + assert_eq!( + report.entities()[0].status, + BindingStatus::Bound { + catalog_name: "GAMMA (5550000001)".to_string(), + basis: BindingBasis::Identifier, + } + ); +} + +#[test] +fn an_identifier_inside_both_names_binds_without_a_hint() { + let catalog = ledgers(&["GAMMA (5550000001)", "GAMMA ALPHA"]); + let binding = bind_one_name(&catalog, "5550000001 GAMMA EPSILON"); + assert_eq!(binding.bound_name(), Some("GAMMA (5550000001)")); +} + +#[test] +fn a_punctuated_stock_code_binds_to_its_unpunctuated_form() { + let catalog = MasterCatalog::new( + MasterClass::StockItem, + ["PH-01A-B00", "PH-02A-B00", "Labour Placeholder"], + ) + .expect("valid"); + let binding = bound(&catalog, &[entity("PH01AB00")]) + .entities() + .first() + .cloned() + .expect("one binding"); + assert_eq!( + binding.status, + BindingStatus::Bound { + catalog_name: "PH-01A-B00".to_string(), + basis: BindingBasis::Identifier, + } + ); +} + +// --- what refuses to bind -------------------------------------------------- + +#[test] +fn one_identifier_carried_by_two_masters_is_ambiguous_never_a_bind() { + let catalog = ledgers(&["ALPHA (5550000001)", "BETA (5550000001)", "GAMMA Supply"]); + let binding = bind_one_name(&catalog, "PARTY 5550000001"); + assert_eq!(reason(&binding), UnboundReason::IdentifierConflict); + assert_eq!( + candidate_names(&binding), + ["ALPHA (5550000001)", "BETA (5550000001)"] + ); +} + +#[test] +fn an_identifier_and_an_exact_name_pointing_apart_is_shown_not_decided() { + let catalog = ledgers(&["ALPHA (5550000001)", "BETA Supply"]); + let source = + SourceEntity::with_identifier_hints(0, "BETA Supply", ["5550000001"]).expect("valid"); + let report = bound(&catalog, &[source]); + let binding = &report.entities()[0]; + assert_eq!(reason(binding), UnboundReason::IdentifierNameConflict); + assert!(candidate_names(binding).contains(&"ALPHA (5550000001)")); + assert!(candidate_names(binding).contains(&"BETA Supply")); +} + +#[test] +fn near_duplicate_masters_produce_candidates_and_choose_none() { + // Three masters differing by one character and word order. The operator + // decides; the module only shows the field. + let catalog = ledgers(&["ALPHA SALE", "ALPHA SALES", "SALES - ALPHA", "Beta Supply"]); + let binding = bind_one_name(&catalog, "ALPHA"); + assert_eq!(reason(&binding), UnboundReason::NearMiss); + assert_eq!( + candidate_names(&binding), + ["ALPHA SALE", "ALPHA SALES", "SALES - ALPHA"] + ); + let unresolved = binding.unresolved().expect("unbound"); + assert_eq!( + unresolved + .candidates + .listed() + .iter() + .map(|candidate| candidate.rule) + .collect::>(), + [ + CandidateRule::CatalogPrefix, + CandidateRule::CatalogPrefix, + CandidateRule::SharedToken + ] + ); + assert_eq!(unresolved.candidates.found(), 3); + assert!(!unresolved.candidates.is_incomplete()); +} + +#[test] +fn a_single_candidate_still_does_not_bind() { + // Four near-misses of ledgers that already existed rejected 61 vouchers. + // Uniqueness of a guess is not evidence. + let catalog = ledgers(&["ALPHA TRADING COMPANY", "Beta Supply"]); + let binding = bind_one_name(&catalog, "ALPHA TRADING COMP"); + assert_eq!(reason(&binding), UnboundReason::NearMiss); + assert_eq!(candidate_names(&binding), ["ALPHA TRADING COMPANY"]); + assert_eq!(binding.bound_name(), None); +} + +#[test] +fn a_truncated_source_name_surfaces_the_longer_master() { + let catalog = ledgers(&["DELTA WHOLESALE PLACEHOLDER", "Beta Supply"]); + let binding = bind_one_name(&catalog, "DELTA WHOLESALE PL"); + assert_eq!( + binding + .unresolved() + .expect("unbound") + .candidates + .listed() + .first() + .map(|candidate| candidate.rule), + Some(CandidateRule::CatalogPrefix) + ); +} + +#[test] +fn a_source_name_extending_a_master_surfaces_the_shorter_master() { + let catalog = ledgers(&["DELTA WHOLESALE", "Beta Supply"]); + let binding = bind_one_name(&catalog, "DELTA WHOLESALE PLACEHOLDER BRANCH"); + let unresolved = binding.unresolved().expect("unbound"); + assert!(unresolved + .candidates + .listed() + .iter() + .any(|candidate| candidate.rule == CandidateRule::SourcePrefix + && candidate.catalog_name == "DELTA WHOLESALE")); +} + +#[test] +fn nothing_defensible_is_unmatched_with_no_candidate() { + let catalog = ledgers(&["Alpha Traders", "Beta Supply"]); + let binding = bind_one_name(&catalog, "Zeta Placeholder"); + assert!(matches!(binding.status, BindingStatus::Unmatched(_))); + assert_eq!(reason(&binding), UnboundReason::NoCandidate); + assert!(candidate_names(&binding).is_empty()); + assert_eq!( + reason(&binding).safe_reason_code(), + "master_binding_no_candidate" + ); +} + +// --- the identifier rules fail closed -------------------------------------- + +#[test] +fn a_date_shaped_run_is_not_an_identifier() { + // Two unrelated period-labelled masters must not fuse on their period. + let catalog = ledgers(&["ALPHA 2026-09-10", "BETA 2026-09-10", "Gamma Supply"]); + let binding = bind_one_name(&catalog, "DELTA 2026-09-10"); + assert!(binding.unresolved().is_some()); + assert!(binding + .unresolved() + .expect("unbound") + .unresolved_identity + .is_empty()); +} + +#[test] +fn a_short_digit_run_is_not_an_identifier() { + // A masked last-four cannot bind two accounts that share four digits. + let catalog = ledgers(&["ALPHA BANK CA 2129", "BETA BANK CA 2129"]); + let binding = bind_one_name(&catalog, "GAMMA BANK CA 2129"); + assert!(binding + .unresolved() + .expect("unbound") + .unresolved_identity + .is_empty()); +} + +#[test] +fn separated_digit_groups_do_not_fuse_into_an_identifier() { + let entity = entity("ALPHA 5550 0000 01"); + assert!(entity.identifiers().is_empty()); +} + +// --- the review findings, pinned ------------------------------------------- + +#[test] +fn a_byte_exact_name_carrying_a_number_is_reported_exact_not_identifier() { + // The write gate admits `ExactName` only. Reporting `Identifier` when the + // two agree made every ledger with a number in its name permanently + // unimportable — the exact population this contract exists to serve. + let catalog = ledgers(&["GAMMA (5550000001)", "GAMMA ALPHA"]); + let binding = bind_one_name(&catalog, "GAMMA (5550000001)"); + assert_eq!( + binding.status, + BindingStatus::Bound { + catalog_name: "GAMMA (5550000001)".to_string(), + basis: BindingBasis::ExactName, + } + ); +} + +#[test] +fn a_byte_exact_name_binds_even_when_its_identifier_is_shared() { + // Found by seeding two live ledgers that share an embedded number, which no + // fabricated fixture had combined. Refusing a name that exactly names one + // master makes that ledger permanently unimportable. + let catalog = ledgers(&[ + "MB PARTY DELTA (5550001009)", + "MB PARTY EPSILON (5550001009)", + "Beta Supply", + ]); + let binding = bind_one_name(&catalog, "MB PARTY DELTA (5550001009)"); + assert_eq!( + binding.status, + BindingStatus::Bound { + catalog_name: "MB PARTY DELTA (5550001009)".to_string(), + basis: BindingBasis::ExactName, + } + ); + // The shared identifier alone, with no exact name, still refuses. + let source = + SourceEntity::with_identifier_hints(0, "SOME PARTY", ["5550001009"]).expect("valid"); + let report = bound(&catalog, &[source]); + assert_eq!( + report.entities()[0].unresolved().expect("unbound").reason, + UnboundReason::IdentifierConflict + ); +} + +#[test] +fn a_decisive_identifier_pointing_elsewhere_still_outranks_a_byte_exact_name() { + let catalog = ledgers(&["ALPHA (5550000001)", "BETA Supply"]); + let source = + SourceEntity::with_identifier_hints(0, "BETA Supply", ["5550000001"]).expect("valid"); + let report = bound(&catalog, &[source]); + assert_eq!( + report.entities()[0].unresolved().expect("unbound").reason, + UnboundReason::IdentifierNameConflict + ); +} + +#[test] +fn space_hyphen_and_slash_are_one_separator_in_both_directions() { + // `TALLY_PROTOCOL_REFERENCE.md` §9.4d, measured on licensed TallyPrime 7.1 + // by naming each spelling in a voucher and reading the day book back to see + // which master it reached. Both hyphen directions matched, and so did a + // slash — so this fold is symmetric, and one key per side is enough. + for (master, source) in [ + ("BRIDGE-PROBE-LEDGER-A", "BRIDGE PROBE LEDGER A"), + ("BRIDGE PROBE LEDGER A", "BRIDGE-PROBE-LEDGER-A"), + ("BRIDGE PROBE LEDGER A", "BRIDGE/PROBE/LEDGER/A"), + ("BRIDGE/PROBE/LEDGER/A", "BRIDGE-PROBE-LEDGER-A"), + ] { + let catalog = ledgers(&[master, "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, source).bound_name(), + Some(master), + "{source} did not reach {master}" + ); + } + + // `X - Y` is a common ledger convention, and it needs the separator step + // and the whitespace-run step together. Both are measured, so it resolves. + let spaced_hyphen = ledgers(&["Bank - HDFC Current", "Beta Supply"]); + assert_eq!( + bind_one_name(&spaced_hyphen, "Bank HDFC Current").bound_name(), + Some("Bank - HDFC Current") + ); + + // An en dash and an underscore were **sent and rejected**. They are not + // separators to Tally, however much they look like them, so they may only + // suggest — this is the half of §9.4d that a "normalises separators" + // reading would get wrong in the dangerous direction. + for (master, source) in [ + ("Alpha \u{2013} Traders", "Alpha Traders"), + ("Alpha_Traders", "Alpha Traders"), + ] { + let catalog = ledgers(&[master, "Beta Supply"]); + let binding = bind_one_name(&catalog, source); + assert_eq!( + binding.bound_name(), + None, + "{source} resolved onto {master} on an equivalence Tally rejects" + ); + assert_eq!(candidate_names(&binding), [master]); + } +} + +#[test] +fn masters_that_collapse_under_the_fold_are_refused_never_chosen() { + // `TALLY_PROTOCOL_REFERENCE.md` §9.4b requires prefer-exact, + // refuse-ambiguous, never-pick. `A-B` and `A B` collapse under the three + // verified transformations and nothing measured says which one Tally would + // choose, so a fold that returns the first match is the failure mode. + let catalog = ledgers(&["Alpha-Beta", "Alpha Beta", "Gamma"]); + + // Prefer-exact: byte equality outranks a key shared by two masters. + assert_eq!( + bind_one_name(&catalog, "Alpha Beta").bound_name(), + Some("Alpha Beta") + ); + assert_eq!( + bind_one_name(&catalog, "Alpha-Beta").bound_name(), + Some("Alpha-Beta") + ); + + // Refuse-ambiguous, never-pick: with no exact spelling to prefer, the + // collapse is reported with both masters offered, not resolved to one. + // Both spellings answer to one key now that §9.4d has measured the hyphen + // in both directions, so a third spelling reaching both is an ambiguity — + // and Tally agrees, because it would match that name to either. + for spelling in ["alpha beta", "ALPHA BETA", "alpha/beta"] { + let binding = bind_one_name(&catalog, spelling); + assert_eq!(reason(&binding), UnboundReason::NameAmbiguous, "{spelling}"); + assert_eq!(binding.bound_name(), None); + assert_eq!(candidate_names(&binding), ["Alpha Beta", "Alpha-Beta"]); + } +} + +#[test] +fn the_master_fold_stops_where_tally_stops() { + // `IMPLEMENTATION_GUIDE.md` §3.3b also measured what Tally does NOT + // normalise: `AND` for `&`, a missing suffix word, and a singular for a + // plural were all rejected. The first two were re-measured on licensed + // 7.1 in §9.4d against `Profit & Loss A/c` and rejected there too. + // Folding further than the authority would bind names Tally refuses. + let catalog = ledgers(&["ZZ Ram & Sons Pvt Ltd", "Beta Supply"]); + for wrong in [ + "ZZ Ram AND Sons Pvt Ltd", + "ZZ Ram & Sons", + "ZZ Ram & Son Pvt Ltd", + ] { + assert_eq!( + bind_one_name(&catalog, wrong).bound_name(), + None, + "{wrong:?} bound, but Tally rejects it" + ); + } + + // The *absent-master* direction, which prefer-exact and refuse-ambiguous + // do not cover: the requested master is not in the book and one different + // ledger collapses onto the request, so there is one candidate and no + // ambiguity to refuse. Uniqueness under a fold is only as meaningful as + // the fold, and `&` has to stay significant for this to hold. + for (requested, present) in [("A & B", "AB"), ("AB", "A & B")] { + let only = ledgers(&[present, "Gamma"]); + assert_eq!( + bind_one_name(&only, requested).bound_name(), + None, + "{requested:?} bound to {present:?}, which Tally treats as a different master" + ); + } +} + +#[test] +fn a_trailing_space_never_claims_byte_equality() { + // `Bank ` against live `Bank` must not report exact: the import file would + // still carry the trailing space. Normalized is the correct, loud outcome — + // the write gate refuses it. + let catalog = ledgers(&["Bank"]); + let binding = bind_one_name(&catalog, "Bank "); + assert_eq!( + binding.status, + BindingStatus::Bound { + catalog_name: "Bank".to_string(), + basis: BindingBasis::NormalizedName, + } + ); + assert_eq!( + binding.source_name, "Bank ", + "the requested value is echoed verbatim" + ); +} + +#[test] +fn digits_inside_a_mixed_code_are_not_also_a_standalone_identifier() { + // Otherwise `Part AB12345678` collides with an unrelated `Bank 12345678`. + let entity = entity("Part AB12345678"); + assert_eq!( + entity.identifiers(), + [Identifier { + kind: IdentifierKind::Code, + value: "AB12345678".to_string(), + }] + ); + let catalog = ledgers(&["Bank 12345678", "Beta Supply"]); + let binding = bind_one_name(&catalog, "Part AB12345678"); + assert_eq!( + binding.bound_name(), + None, + "a part code must not reach a bank ledger" + ); +} + +#[test] +fn a_fiscal_period_label_is_not_an_identity_bearing_code() { + // Two unrelated ledgers routinely share a period label. Identifier-first + // matching would otherwise bind the source to whichever one exists before + // it ever compared the names. + for label in [ + "FY25", + "FY2025", + "AY2026", + "Q3", + "H2", + "PER2026", + "APR2025", + "2025Q1", + "MAR26", + "H12026", + // A month name can be any length; a year cannot. Capping the + // alphabetic run kept losing to longer spellings. + "SEPTEMBER2025", + "2025QUARTER1", + "DECEMBER2026", + // Ranges: operators write these with the very separators that + // canonicalization strips, so the test has to see the raw token. + "FY2025-26", + "FY2025/26", + // The comparison key folds these dash variants; the period boundary + // has to admit the same set or the range fuses instead of splitting. + "FY2025\u{2013}26", + "FY2025\u{2014}26", + "2025\u{2013}2026", + "2025-2026", + "APR2025-MAR2026", + // Written without the separator, the range arrives as one run that + // every length test above missed, and the token passed as a code. + "FY202425", + "FY20242025", + "AY202526", + "202425FY", + ] { + assert!( + entity(&format!("Purchases {label}")) + .identifiers() + .is_empty(), + "{label} was treated as a code identifier" + ); + } + for label in ["FY2025", "FY202425"] { + let catalog = ledgers(&[&format!("Sales {label}"), "Beta Supply"]); + let binding = bind_one_name(&catalog, &format!("Purchases {label}")); + assert_eq!( + binding.bound_name(), + None, + "the shared period label {label} bound two unrelated ledgers" + ); + } + // A genuine identity-bearing code still is one. + assert_eq!(entity("Item PH01AB00").identifiers().len(), 1); +} + +#[test] +fn a_name_in_another_script_does_not_shed_its_letters_into_a_code() { + // Canonicalization keeps only ASCII, so a Devanagari party name fused to an + // ASCII suffix yielded the code `AB12345678` — a string the name never + // contained — and reached an unrelated bank ledger. The ASCII spelling of + // the same shape never did, which is what makes it a defect rather than a + // policy: the boundary was an ASCII boundary wearing a general name. + let party = "\u{92a}\u{93e}\u{930}\u{94d}\u{91f}\u{940}"; + let fused = format!("{party}AB12345678"); + assert!( + entity(&fused).identifiers().is_empty(), + "a dropped non-ASCII prefix manufactured a code" + ); + let catalog = ledgers(&["Bank AB12345678", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, &fused).bound_name(), + None, + "a party name must not reach a bank ledger by shedding its script" + ); + // The ASCII spelling this is measured against, unchanged: it keeps every + // letter, so it carries a code of its own and reaches no bank. + assert_eq!( + entity("PartyAB12345678").identifiers(), + [Identifier { + kind: IdentifierKind::Code, + value: "PARTYAB12345678".to_string(), + }] + ); + assert_eq!( + bind_one_name(&catalog, "PartyAB12345678").bound_name(), + None + ); + // A code standing on its own beside a name in any script is still a code. + assert_eq!( + entity(&format!("{party} AB12345678")).identifiers().len(), + 1 + ); + + // Letters were the first guard and were too narrow: `char::is_alphabetic` + // is false for a Devanagari digit, so non-ASCII numerals walked through it + // and canonicalization dropped them just the same. The numeric branch had + // the identical hole, which no thread named — a trailing run of Devanagari + // digits is not alphabetic either, so the ASCII digits before it were + // emitted as a whole account number. + let digits = "\u{967}\u{968}\u{969}"; + for fused in [ + format!("Purchases AB{digits}12345678"), + format!("Purchases 12345678{digits}"), + format!("Purchases {digits}12345678"), + ] { + assert!( + entity(&fused).identifiers().is_empty(), + "{fused} manufactured an identifier out of what canonicalization dropped" + ); + } + let bank = ledgers(&["Bank AB12345678", "Bank 12345678", "Beta Supply"]); + assert_eq!( + bind_one_name(&bank, &format!("Purchases AB{digits}12345678")).bound_name(), + None + ); + assert_eq!( + bind_one_name(&bank, &format!("Purchases 12345678{digits}")).bound_name(), + None + ); + // Only the separators §9.4d measured are discarded from a code. An ASCII + // hyphen and slash are; a non-breaking hyphen is not, because §9.4d sent a + // non-ASCII dash and watched Tally reject it. So a code punctuated with one + // no longer agrees with the plain spelling — a refusal, which is the safe + // direction, and the token is still admitted as a candidate by name. + assert_eq!( + entity("Item PH-01/AB-00").identifiers(), + entity("Item PH01AB00").identifiers() + ); + assert_ne!( + entity("Item PH\u{2011}01AB00").identifiers(), + entity("Item PH01AB00").identifiers() + ); +} + +#[test] +fn two_encodings_of_one_name_are_two_masters_to_tally_and_so_to_this() { + // Measured 2026-08-19 on TallyPrime 7.1: a voucher naming a UI-created + // ledger in its canonically equivalent NFD spelling was rejected with + // `EXCEPTIONS=1` and a LINEERROR saying the ledger does not exist, while + // the NFC spelling created it. Tally stores the bytes it was given and + // matches on exact codepoints, so these are different masters to Tally and + // must be different masters here. + // + // This is stronger than the UNVERIFIED rows in §9.4b's table: folding it + // is not unproven, it is proven wrong. It reads like decoding rather than + // folding, which is why it nearly stayed in the resolving fold. + let precomposed = "Caf\u{e9} Traders"; + let decomposed = "Cafe\u{301} Traders"; + let catalog = ledgers(&[precomposed, "Beta Supply"]); + let binding = bind_one_name(&catalog, decomposed); + assert_eq!( + binding.bound_name(), + None, + "an NFD source name resolved onto an NFC master Tally keeps apart" + ); + // The wide fold still reaches it, so an operator sees the one master worth + // looking at rather than nothing at all. + assert_eq!(candidate_names(&binding), [precomposed]); + // And the spelling Tally would actually match still binds. + assert_eq!( + bind_one_name(&catalog, precomposed).bound_name(), + Some(precomposed) + ); +} + +#[test] +fn a_date_fused_into_a_code_shaped_token_is_still_a_date() { + // `is_plausible_date` guarded the numeric branch only, and `is_period` + // never sees this one: its eight-digit case admits a year followed by a + // year, and `0911` is neither. So `DATED20250911` cleared every code test + // and two unrelated ledgers bound to each other on a shared date label. + for label in [ + "DATED20250911", + "DT20250911", + "INV20250911", + "DATED11092025", + ] { + assert!( + entity(&format!("Purchases {label}")) + .identifiers() + .is_empty(), + "{label} was treated as a code identifier" + ); + } + let catalog = ledgers(&["Sales DATED20250911", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, "Purchases DATED20250911").bound_name(), + None, + "a shared date label must not bind two unrelated ledgers" + ); + // A run no calendar would produce is still a code, and a longer run is not + // a date at all — the test is on eight digits exactly. + assert_eq!(entity("Item PH01AB00").identifiers().len(), 1); + assert_eq!(entity("Party AB5550001001").identifiers().len(), 1); +} + +#[test] +fn a_catalog_is_bounded_by_total_bytes_and_not_only_by_count() { + // Both documented bounds can hold while their product does not: 20,000 + // names of 16,384 characters satisfies each and is 327 MB before the + // constructor builds keys, tokens and four indexes over them. The bound has + // to be on the aggregate, and has to fire while the iterator is consumed. + let long = "N".repeat(MAX_NAME_CHARS); + let many = (0..600) + .map(|index| format!("{index:04}{}", &long[..MAX_NAME_CHARS - 4])) + .collect::>(); + assert_eq!( + MasterCatalog::new(MasterClass::Ledger, &many), + Err(MasterBindingError::CatalogTooLarge) + ); + // A catalog far larger than any observed book still loads: the largest read + // here is 470 names, so the bound must not be reachable in practice. + let ordinary = (0..5_000) + .map(|index| format!("Placeholder Party {index:05}")) + .collect::>(); + assert!(MasterCatalog::new(MasterClass::Ledger, &ordinary).is_ok()); +} + +#[test] +fn repeating_one_source_name_does_not_repeat_the_search_or_change_the_answer() { + // A draft may name one ledger on every row, and the candidate search is not + // cheap when the name reaches a family. Remembering it must not change what + // the report says — the memo is keyed on the source key and the masters its + // identifiers reached, which is all `collect_candidates` reads. + let names = (0..60) + .map(|index| format!("Acme Branch {index:05}")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + + let alone = bind_one_name(&catalog, "Acme Branch"); + let repeated = (0..40) + .map(|position| SourceEntity::new(position, "Acme Branch").expect("valid")) + .collect::>(); + // Counted, not assumed. The first version of this memo was consulted on the + // conflict and ambiguity paths but not on the ordinary near miss — which is + // the case this test uses — and every assertion below still passed, because + // they check the answer rather than the work. `Acme Branch` matches no + // master exactly, by identifier or by the narrow fold, so all forty entities + // take that path. + super::CANDIDATE_SEARCHES.with(|count| count.set(0)); + let report = bound(&catalog, &repeated); + assert_eq!( + super::CANDIDATE_SEARCHES.with(std::cell::Cell::get), + 1, + "forty rows naming one ledger ran the candidate search more than once" + ); + assert_eq!(report.totals().requested, 40); + assert_eq!(report.totals().bound, 0); + for entity in report.entities() { + assert_eq!( + entity.status, alone.status, + "remembering the search changed the answer" + ); + } + + // Same key, different spelling: one is byte-exact and one is not, and the + // shared memo must not leak the exact hit into the other's answer. + let mixed = vec![ + SourceEntity::new(0, "Acme Branch 00007").expect("valid"), + SourceEntity::new(1, "acme branch 00007").expect("valid"), + ]; + let report = bound(&catalog, &mixed); + assert_eq!(report.entities()[0].bound_name(), Some("Acme Branch 00007")); + assert_eq!( + report.entities()[1].bound_name(), + Some("Acme Branch 00007"), + "a normalized hit is still a hit" + ); + + // Same source *name*, different identifier hints. The key is identical, so + // a memo keyed on the key alone would hand the second entity the first + // one's candidates — a different pair of ledgers entirely. This is the case + // that proves the second half of the memo key, and nothing else reaches it. + let shared = ledgers(&[ + "Party Alpha (5550001009)", + "Party Beta (5550001009)", + "Party Gamma (5550001007)", + "Party Delta (5550001007)", + ]); + let hinted = vec![ + SourceEntity::with_identifier_hints(0, "Zeta Holdings", ["5550001009"]).expect("valid"), + SourceEntity::with_identifier_hints(1, "Zeta Holdings", ["5550001007"]).expect("valid"), + ]; + let report = bound(&shared, &hinted); + assert_eq!( + candidate_names(&report.entities()[0]), + ["Party Alpha (5550001009)", "Party Beta (5550001009)"] + ); + assert_eq!( + candidate_names(&report.entities()[1]), + ["Party Delta (5550001007)", "Party Gamma (5550001007)"], + "the memo handed one entity another's candidates" + ); +} + +#[test] +fn a_retained_identity_stays_short_enough_to_write_back() { + // An unresolved entity carries its identifiers into a fallback so the money + // can be found later, and the documented way to carry them is a narration — + // which the import path refuses over 2,000 characters. Unbounded values let + // `assign_fallback` succeed while producing a tag nobody could write, which + // fails at the write rather than here. + let long_run = "5".repeat(400); + assert!( + entity(&format!("Party {long_run}")) + .identifiers() + .is_empty(), + "a 400-digit run is not an account number" + ); + let long_code = format!("AB{}", "7".repeat(400)); + assert!(entity(&format!("Party {long_code}")) + .identifiers() + .is_empty()); + + // The bound is on the value, so the tag stays complete rather than + // truncated — a truncated identity is worse than none, because it looks + // usable. Thirty-two of the longest admitted identifiers still fit. + let hints = (0..MAX_IDENTIFIERS_PER_NAME) + .map(|index| format!("{index:02}{}", "5".repeat(MAX_IDENTIFIER_CHARS - 2))) + .collect::>(); + let source = + SourceEntity::with_identifier_hints(0, "Zeta Holdings", hints.iter().map(String::as_str)) + .expect("the longest admitted identifiers are still admitted"); + let catalog = ledgers(&["Alpha Traders", "Beta Supply"]); + let report = bound(&catalog, &[source]); + let tag = report + .assign_fallback(0, &catalog, "Alpha Traders") + .expect("a fallback in the same catalog") + .retained_tag(); + assert!( + tag.len() <= 2_000, + "a retained identity of {} characters cannot be written back", + tag.len() + ); + // Complete, not truncated: every identifier is still in it. + assert_eq!(tag.matches("numeric:").count(), MAX_IDENTIFIERS_PER_NAME); +} + +#[test] +fn an_identifier_held_by_a_whole_family_is_a_conflict_without_expanding_it() { + // One identifier on more masters than a candidate list may show is already + // a conflict, and its holders are a family this entity does not separate. + // Building the set anyway cloned it per source row, before the candidate + // memo was consulted — the cost is paid on a result nothing can use. + let names = (0..MAX_CANDIDATES_PER_ENTITY + 5) + .map(|index| format!("Shared Party {index:03} (5550009999)")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + // Counted, not inferred. Refusing to expand and expanding then refusing + // produce the same verdict, so only a count can tell them apart. + super::HOLDER_EXPANSIONS.with(|count| count.set(0)); + let binding = bind_one_name(&catalog, "Zeta Holdings 5550009999"); + assert_eq!( + super::HOLDER_EXPANSIONS.with(std::cell::Cell::get), + 0, + "a family larger than any candidate list was expanded anyway" + ); + assert_eq!( + binding.bound_name(), + None, + "a shared identifier never binds" + ); + assert_eq!(reason(&binding), UnboundReason::IdentifierConflict); + // The identity is still reported, so the operator can still find the money. + assert_eq!( + binding.unresolved().expect("unbound").unresolved_identity, + [Identifier { + kind: IdentifierKind::Numeric, + value: "5550009999".to_string(), + }] + ); +} + +#[test] +fn a_hint_pointing_elsewhere_outranks_an_exact_name_at_any_family_size() { + // The large-holder skip added for cost made this invariant size-dependent: + // below the cap the holder set was built and `identifier_points_elsewhere` + // saw that it did not contain the exact master; above the cap the set was + // skipped, the signal went with it, and the exact name bound while the + // hint pointed entirely elsewhere. Skipping the expansion must not skip + // the question the expansion was asked. + let mut names = vec!["Alpha Traders".to_string()]; + names.extend( + (0..MAX_CANDIDATES_PER_ENTITY + 5) + .map(|index| format!("Other Party {index:03} (5550008888)")), + ); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let source = + SourceEntity::with_identifier_hints(0, "Alpha Traders", ["5550008888"]).expect("valid"); + let report = bound(&catalog, &[source]); + let binding = &report.entities()[0]; + assert_eq!( + binding.bound_name(), + None, + "a byte-exact name bound while its hint pointed at a different family" + ); + assert_eq!(reason(binding), UnboundReason::IdentifierNameConflict); + + // Below the cap the same shape already behaved; both sides of the boundary + // are asserted so the fix cannot regress on one of them alone. + let mut small = vec!["Alpha Traders".to_string()]; + small.extend((0..3).map(|index| format!("Other Party {index:03} (5550008888)"))); + let small = MasterCatalog::new(MasterClass::Ledger, &small).expect("valid"); + let source = + SourceEntity::with_identifier_hints(0, "Alpha Traders", ["5550008888"]).expect("valid"); + let report = bound(&small, &[source]); + assert_eq!( + reason(&report.entities()[0]), + UnboundReason::IdentifierNameConflict + ); +} + +#[test] +fn a_date_range_is_dates_even_after_its_separator_is_removed() { + // `20250911-20250912` fuses to sixteen digits, which is no length + // `is_plausible_date` recognizes, and `is_period` reads neither half as a + // year range. So a date *range* walked through a guard a single date does + // not — the fusing is what hid the components, so they are checked first. + for range in [ + "20250911-20250912", + "20250911/20250912", + "01012026-02012026", + "20250911-20250912-20250913", + ] { + assert!( + entity(&format!("Purchases {range}")) + .identifiers() + .is_empty(), + "{range} was treated as an identifier" + ); + } + let catalog = ledgers(&["Sales 20250911-20250912", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, "Purchases 20250911-20250912").bound_name(), + None, + "a shared date range must not bind two unrelated ledgers" + ); + // A punctuated account number is untouched: no component reads as a date. + assert_eq!(entity("Party 5550001-002").identifiers().len(), 1); +} + +#[test] +fn a_stock_item_may_suggest_on_a_fold_but_not_resolve_on_one() { + // §9.4d measured **ledgers**. Whether stock items match by the same rule + // was never sent, so the same folded pair that resolves for a ledger may + // only be offered for a stock item. + let folded = ["Sales-Item", "Beta Supply"]; + let ledger = MasterCatalog::new(MasterClass::Ledger, folded).expect("valid"); + assert_eq!( + bind_one_name(&ledger, "sales item").bound_name(), + Some("Sales-Item") + ); + + let items = MasterCatalog::new(MasterClass::StockItem, folded).expect("valid"); + let source = SourceEntity::new(0, "sales item").expect("valid"); + let report = bind(&items, &[source]).expect("valid"); + let binding = &report.entities()[0]; + assert_eq!( + binding.bound_name(), + None, + "a stock item resolved on an unmeasured fold" + ); + assert_eq!(candidate_names(binding), ["Sales-Item"]); + // A lone unlicensed fold is a **near miss**, not an ambiguity. Nothing + // shares its key; one master simply did not qualify, and `NameAmbiguous` + // would tell a consumer several masters collided — a different fact with a + // different remedy. + assert_eq!(reason(binding), UnboundReason::NearMiss); + + // Byte equality needs no fold and is unaffected by the class. + let exact = SourceEntity::new(0, "Sales-Item").expect("valid"); + let report = bind(&items, &[exact]).expect("valid"); + assert_eq!(report.entities()[0].bound_name(), Some("Sales-Item")); +} + +#[test] +fn a_repeated_key_is_remembered_however_many_distinct_ones_precede_it() { + // The entry cap made the memo's protection depend on **source order**: + // enough distinct cheap misses at the head of a draft filled it, and the + // repeated expensive key behind them was then never cached — the stall the + // memo exists to prevent, reachable by reordering the same rows. + let names = (0..60) + .map(|index| format!("Acme Branch {index:05}")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + + let mut entities = (0..MAX_CANDIDATE_MEMO_ENTRIES) + .map(|index| SourceEntity::new(index, &format!("Distinct Miss {index:05}")).expect("valid")) + .collect::>(); + entities.extend((0..4).map(|offset| { + SourceEntity::new(MAX_CANDIDATE_MEMO_ENTRIES + offset, "Acme Branch").expect("valid") + })); + + super::CANDIDATE_SEARCHES.with(|count| count.set(0)); + let report = bound(&catalog, &entities); + let searches = super::CANDIDATE_SEARCHES.with(std::cell::Cell::get); + assert_eq!(report.totals().requested, MAX_CANDIDATE_MEMO_ENTRIES + 4); + // One search for the repeated key, not four. The distinct misses each cost + // one of their own, so the total is bounded by the distinct count plus one. + assert!( + searches <= MAX_CANDIDATE_MEMO_ENTRIES + 1, + "the repeated key was searched more than once: {searches} searches" + ); +} + +#[test] +fn unmeasured_punctuation_keeps_two_codes_apart() { + // Canonicalization filtered to alphanumerics, so **every** ASCII + // punctuation mark was discarded and `AB_123456` canonicalized the same as + // `AB-123456` — while §9.4d had sent an underscore at a live master and + // watched Tally reject it. The fold's evidence is about hyphens and + // slashes; everything else is content. + let catalog = ledgers(&["Sales AB-123456", "Beta Supply"]); + let binding = bind_one_name(&catalog, "Purchases AB_123456"); + assert_eq!( + binding.bound_name(), + None, + "an underscore was treated as a hyphen on evidence that says it is not" + ); + // The measured separators still agree, and the contrast is the point: with + // a hyphen the two codes are one identifier and the bind is the + // identifier-first rule working; with an underscore they are two + // identifiers and nothing binds. + let measured = ledgers(&["Sales PH-01-AB-00", "Beta Supply"]); + assert_eq!( + bind_one_name(&measured, "Purchases PH01AB00").bound_name(), + Some("Sales PH-01-AB-00"), + "a hyphen and no hyphen are one code, which §9.4d measured" + ); + assert_eq!( + entity("Item PH-01-AB-00").identifiers(), + entity("Item PH01AB00").identifiers() + ); + assert_ne!( + entity("Item AB_123456").identifiers(), + entity("Item AB-123456").identifiers() + ); +} + +#[test] +fn a_withheld_family_still_reports_how_many_share_the_identifier() { + // Skipping the expansion discarded the holder count with the set, so a + // withheld family reported `found() == 0` and an empty listing — telling + // the operator nothing shares the identifier when hundreds do. The count is + // the one thing a reader still needs from a set too large to show. + let names = (0..MAX_CANDIDATES_PER_ENTITY + 7) + .map(|index| format!("Shared Party {index:03} (5550007777)")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let source = + SourceEntity::with_identifier_hints(0, "Zeta Holdings", ["5550007777"]).expect("valid"); + let report = bound(&catalog, &[source]); + let unresolved = report.entities()[0].unresolved().expect("unbound"); + assert_eq!(unresolved.reason, UnboundReason::IdentifierConflict); + assert_eq!( + unresolved.candidates.found(), + MAX_CANDIDATES_PER_ENTITY + 7, + "a withheld family reported no holders at all" + ); + assert!( + unresolved.candidates.is_incomplete(), + "a count without a listing must say the listing is incomplete" + ); +} + +#[test] +fn a_report_bounds_its_own_candidate_allocation() { + // A per-entity cap does not bound a report: the clones exist the moment it + // is built, and a consumer capping its own copy afterwards bounds only the + // copy. The budget is spent in entity order; entities past it keep their + // true count and flag truncation. + let long = "Z".repeat(400); + let names = (0..30) + .map(|index| format!("SHARED PREFIX {index:03} {long}")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let entities = (0..2_000) + .map(|position| SourceEntity::new(position, "SHARED PREFIX 001").expect("valid")) + .collect::>(); + let report = bound(&catalog, &entities); + let listed: usize = report + .unbound() + .filter_map(|entity| entity.unresolved()) + .map(|unresolved| { + unresolved + .candidates + .listed() + .iter() + .map(|candidate| candidate.catalog_name.len()) + .sum::() + }) + .sum(); + assert!( + listed <= MAX_REPORT_CANDIDATE_BYTES, + "report allocated {listed} candidate bytes" + ); + let starved = report + .unbound() + .filter_map(|entity| entity.unresolved()) + .filter(|unresolved| unresolved.candidates.listed().is_empty()) + .collect::>(); + assert!(!starved.is_empty(), "the budget must actually bite here"); + assert!(starved.iter().all( + |unresolved| unresolved.candidates.found() > 0 && unresolved.candidates.is_incomplete() + )); +} + +#[test] +fn a_token_carrying_letters_never_yields_a_standalone_number() { + // A one-letter token fails the code test, and its digits were then escaping + // as a numeric of their own — so `Part A12345678` could reach an unrelated + // `Bank 12345678`. A token identifies by its whole shape or not at all. + assert!(entity("Part A12345678").identifiers().is_empty()); + let catalog = ledgers(&["Bank 12345678", "Beta Supply"]); + assert_eq!(bind_one_name(&catalog, "Part A12345678").bound_name(), None); + // A bare digit run beside no letters is still an identifier. + assert_eq!(entity("Party (5550001001)").identifiers().len(), 1); +} + +#[test] +fn conflicting_identifiers_outrank_a_byte_exact_name_but_a_shared_one_does_not() { + // Two hints selecting two other masters is conflicting evidence, and + // binding the name silently discarded it. A *shared* identifier is + // different: the ambiguous set still contains the master the name spells, + // so the name is what separates it from its siblings. + let catalog = ledgers(&["ACME", "BETA 11111111", "GAMMA 22222222"]); + let source = + SourceEntity::with_identifier_hints(0, "ACME", ["11111111", "22222222"]).expect("valid"); + let report = bound(&catalog, &[source]); + let binding = &report.entities()[0]; + assert_eq!(reason(binding), UnboundReason::IdentifierNameConflict); + // Every master the evidence reached is offered, so the operator sees the + // disagreement rather than one side of it — and the byte-exact name leads, + // labelled as itself. This refusal exists *because* byte equality was + // observed, so burying that under the identifier that outranked it left + // the operator reading two facts without being told one of them was exact. + assert_eq!( + candidate_names(binding), + ["ACME", "BETA 11111111", "GAMMA 22222222"] + ); + assert_eq!( + binding.unresolved().expect("unbound").candidates.listed()[0].rule, + CandidateRule::ExactName + ); + + // The shared-identifier case must keep binding: one identifier reached the + // master the name spells along with its sibling, and the name separates + // them. + let shared = ledgers(&[ + "MB PARTY DELTA (5550001009)", + "MB PARTY EPSILON (5550001009)", + ]); + assert_eq!( + bind_one_name(&shared, "MB PARTY DELTA (5550001009)").bound_name(), + Some("MB PARTY DELTA (5550001009)") + ); + + // The mixed case, which a union test answers wrongly: the exact master is + // in the union because its own number is one of the identifiers, while a + // second identifier plainly reaches somewhere else. Provenance per + // identifier is the only thing that separates this from the shared case. + let mixed = ledgers(&["ACME 11111111", "BETA 22222222"]); + let source = + SourceEntity::with_identifier_hints(0, "ACME 11111111", ["22222222"]).expect("valid"); + let report = bound(&mixed, &[source]); + assert_eq!( + reason(&report.entities()[0]), + UnboundReason::IdentifierNameConflict + ); +} + +#[test] +fn an_indic_name_is_not_torn_apart_at_its_joins() { + // `char::is_alphanumeric` is false for a Devanagari virama — the halant + // that joins consonants — and false for a nukta. Splitting on "not + // alphanumeric" cut these names at the joins, so a shared word stopped + // being a shared token. These names are in the books this binder reads. + let catalog = ledgers(&[ + "\u{936}\u{94d}\u{930}\u{940} \u{917}\u{923}\u{947}\u{936} \u{91f}\u{94d}\u{930}\u{947}\u{921}\u{930}\u{94d}\u{938}", + "\u{930}\u{93e}\u{92f} \u{90f}\u{923}\u{94d}\u{921} \u{938}\u{928}\u{94d}\u{938}", + "Beta Supply", + ]); + // The second book's distinctive word, which the virama used to fragment + // away entirely, now reaches its own master. + let binding = bind_one_name(&catalog, "\u{938}\u{928}\u{94d}\u{938}"); + assert!( + candidate_names(&binding) + .iter() + .any(|name| name.contains("\u{930}\u{93e}\u{92f}")), + "a shared Indic word did not surface its master: {:?}", + candidate_names(&binding) + ); +} + +#[test] +fn a_non_ascii_name_beside_digits_is_still_a_name() { + // The observed books carry Devanagari, Tamil and Bengali ledger names. An + // ASCII-only letter guard read `पार्टी12345678` as digits standing alone + // and bound a party to an unrelated bank ledger. + let party = "\u{92a}\u{93e}\u{930}\u{94d}\u{91f}\u{940}12345678"; + assert!(entity(party).identifiers().is_empty()); + let catalog = ledgers(&["Bank 12345678", "Beta Supply"]); + assert_eq!(bind_one_name(&catalog, party).bound_name(), None); +} + +#[test] +fn a_masked_value_identifies_nothing() { + // `XXXXX1234X` clears every length and composition test while carrying only + // a last four that any number of parties share. + assert!(entity("Purchases XXXXX1234X").identifiers().is_empty()); + let catalog = ledgers(&["Sales XXXXX1234X", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, "Purchases XXXXX1234X").bound_name(), + None + ); + // Distinct letters are what an identity-bearing code has and a mask does not. + assert_eq!(entity("Item PH01AB00").identifiers().len(), 1); + + // A mask spelled with punctuation reaches the numeric branch instead, where + // every non-digit is an ordinary delimiter — so `********12345678` split + // cleanly and offered its visible suffix as though it were the account. + for masked in ["Purchases ********12345678", "Purchases ####12345678"] { + assert!( + entity(masked).identifiers().is_empty(), + "{masked} exposed its suffix as an identifier" + ); + } + let punctuated = ledgers(&["Sales ********12345678", "Beta Supply"]); + assert_eq!( + bind_one_name(&punctuated, "Purchases ********12345678").bound_name(), + None + ); + // A mask and the digits it hides are often written apart; that is the same + // statement, and reading tokens independently lost the relationship. + for separated in ["Purchases **** 12345678", "Purchases #### 12345678"] { + assert!( + entity(separated).identifiers().is_empty(), + "{separated} exposed its suffix as an identifier" + ); + } + // A mask spelled with letters is the same statement as one spelled with + // punctuation, and it too is written apart from the digits it hides. Read + // token by token, `XXXX` failed the punctuation test and the visible suffix + // escaped as a whole account number. + for separated in [ + "Purchases XXXX 12345678", + "Purchases XXXXXXXX 12345678", + "Purchases (XXXX) 12345678", + ] { + assert!( + entity(separated).identifiers().is_empty(), + "{separated} exposed its suffix as an identifier" + ); + } + let alphabetic = ledgers(&["Sales XXXX 12345678", "Beta Supply"]); + assert_eq!( + bind_one_name(&alphabetic, "Purchases XXXX 12345678").bound_name(), + None, + "a masked last-eight must not bind two unrelated ledgers" + ); + // A suffix shaped as a code is no less hidden than one shaped as a number. + assert!(entity("Purchases XXXX AB12345678").identifiers().is_empty()); + // A delimiter between the mask and its suffix does not unmask it. Reading + // the state token by token, a `-` reset it and the suffix walked out. + for punctuated in [ + "Purchases XXXX - 12345678", + "Purchases **** / 12345678", + "Purchases XXXX . 12345678", + "Purchases XXXX - - 12345678", + ] { + assert!( + entity(punctuated).identifiers().is_empty(), + "{punctuated} exposed its suffix as an identifier" + ); + } + let separated = ledgers(&["Sales XXXX - 12345678", "Beta Supply"]); + assert_eq!( + bind_one_name(&separated, "Purchases XXXX - 12345678").bound_name(), + None + ); + // An ordinary word after a mask does end it, or nothing downstream of one + // could ever identify anything again. + assert_eq!( + entity("Purchases XXXX Invoice 5550001001") + .identifiers() + .len(), + 1 + ); + // Ordinary words are not masks, however repetitive: only a run of one + // repeated letter is, and one letter alone is an ordinary word. + assert_eq!(entity("Purchases Unit 5550001001").identifiers().len(), 1); + assert_eq!(entity("Purchases A 5550001001").identifiers().len(), 1); + // A mask is a shape, not a list of glyphs. Enumerating four of them lost + // to a dotted and an underscored mask, and would have lost to the next. + for shaped in [ + "Purchases ........12345678", + "Purchases ____ 12345678", + "Purchases ~~~ 12345678", + "Purchases --- 12345678", + ] { + assert!( + entity(shaped).identifiers().is_empty(), + "{shaped} exposed its suffix as an identifier" + ); + } + let dotted = ledgers(&["Sales ........12345678", "Beta Supply"]); + assert_eq!( + bind_one_name(&dotted, "Purchases ........12345678").bound_name(), + None + ); + // Ordinary punctuation around a whole number is not a mask. + assert_eq!(entity("Party (5550001001)").identifiers().len(), 1); + assert_eq!(entity("Party 5550001-002").identifiers().len(), 1); + // Names punctuate; they do not repeat punctuation. Two of a character is + // ordinary noise, so the run has to be longer than an operator's slip. + assert_eq!(entity("S.K. Traders 5550001001").identifiers().len(), 1); + assert_eq!(entity("Party -- 5550001001").identifiers().len(), 1); + // And a number following an ordinary word is untouched. + assert_eq!(entity("Invoice 5550001001").identifiers().len(), 1); +} + +#[test] +fn a_fiscal_year_range_is_a_period_not_an_account_number() { + // `2025-2026` strips to an eight-digit run that no calendar reading + // rejects, and two unrelated ledgers share a fiscal year as routinely as + // they share a month. + for range in ["2025-2026", "2025/2026", "1999-2000"] { + assert!( + entity(&format!("Purchases {range}")) + .identifiers() + .is_empty(), + "{range} was treated as an identifier" + ); + } + let catalog = ledgers(&["Sales 2025-2026", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, "Purchases 2025-2026").bound_name(), + None + ); + // A punctuated account number that is not a year range still binds. + let accounts = ledgers(&["Party 5550001-002", "Beta Supply"]); + assert_eq!( + bind_one_name(&accounts, "Other 5550001002").bound_name(), + Some("Party 5550001-002") + ); +} + +#[test] +fn an_eight_digit_date_in_any_admitted_order_is_not_an_identifier() { + for date in ["20260910", "01012026", "31122026", "12312026"] { + assert!( + entity(&format!("Period {date}")).identifiers().is_empty(), + "{date} was treated as an identifier" + ); + } + // A number that reads as no calendar date at all still is one. + assert_eq!(entity("Party 55500001").identifiers().len(), 1); +} + +#[test] +fn more_identifiers_than_the_bound_is_refused_not_truncated() { + // Keeping the first few can discard the identifier that pointed at a + // different master, turning a conflict into a bind. + let many = (0..MAX_IDENTIFIERS_PER_NAME + 1) + .map(|index| format!("5550{index:04}00")) + .collect::>() + .join(" "); + assert_eq!( + SourceEntity::new(0, &many), + Err(MasterBindingError::TooManyIdentifiers) + ); + assert_eq!( + MasterBindingError::TooManyIdentifiers.safe_reason_code(), + "master_identifiers_too_many" + ); +} + +#[test] +fn the_listing_variant_says_what_an_absent_candidate_means() { + // The three facts that used to share one empty vector, now told apart by + // the type. A consumer matching exhaustively is made to decide each. + let catalog = ledgers(&["Alpha Traders", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, "Zeta Placeholder") + .unresolved() + .expect("unbound") + .candidates, + Candidates::None, + "nothing resembles it" + ); + + let family = (0..MAX_PREFIX_FAMILY + 5) + .map(|index| format!("ALPHAGROUP UNIT {index:02}")) + .collect::>(); + let family = MasterCatalog::new(MasterClass::Ledger, &family).expect("valid"); + assert_eq!( + bind_one_name(&family, "ALPHAGROUP") + .unresolved() + .expect("unbound") + .candidates, + Candidates::Withheld { + found: MAX_PREFIX_FAMILY + 5 + }, + "many exist and none separates them" + ); + + let listed = ledgers(&["ALPHA SALE", "ALPHA SALES", "SALES - ALPHA", "Beta Supply"]); + let binding = bind_one_name(&listed, "ALPHA"); + let candidates = &binding.unresolved().expect("unbound").candidates; + assert!(matches!(candidates, Candidates::Listed { .. })); + assert_eq!(candidates.found(), 3); +} + +#[test] +fn only_an_incomplete_listing_may_withhold_an_absence() { + // The predicate a consumer needs before reporting "nothing like this is + // present". `None` permits that conclusion; the other two forbid it. + assert!(!Candidates::None.is_incomplete()); + assert!(!Candidates::Listed { listed: Vec::new() }.is_incomplete()); + assert!(Candidates::Withheld { found: 30 }.is_incomplete()); + assert!(Candidates::Truncated { + listed: Vec::new(), + found: 9 + } + .is_incomplete()); + // `found` is the total, never the listed length, wherever it is known. + assert_eq!(Candidates::Withheld { found: 30 }.found(), 30); + assert!(Candidates::Withheld { found: 30 }.listed().is_empty()); +} + +// --- candidate discipline -------------------------------------------------- + +#[test] +fn a_catalog_wide_token_stops_discriminating() { + let mut names = (0..40) + .map(|index| format!("PLACEHOLDER UNIT {index:02}")) + .collect::>(); + names.push("ALPHA PLACEHOLDER TRADERS".to_string()); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + // "placeholder" is carried by every entry, so it may not pull all 41 in. + let binding = bind_one_name(&catalog, "PLACEHOLDER ZETA"); + assert!(binding.unresolved().expect("unbound").candidates.found() <= 1); +} + +#[test] +fn a_prefix_matching_a_whole_family_is_counted_and_deliberately_not_listed() { + // Measured against live books: listing an arbitrary capped slice of a name + // family put the right master out of view about a third of the time, + // because the slice is ordered by name and the family is uniform. Counting + // the family and listing none of it is the honest answer — the source name + // genuinely does not distinguish one from another. + let names = (0..MAX_PREFIX_FAMILY + 5) + .map(|index| format!("ALPHAGROUP UNIT {index:02}")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let binding = bind_one_name(&catalog, "ALPHAGROUP"); + let unresolved = binding.unresolved().expect("unbound"); + assert_eq!(reason(&binding), UnboundReason::NoDiscriminatingCandidate); + assert!(unresolved.candidates.listed().is_empty()); + assert_eq!(unresolved.candidates.found(), MAX_PREFIX_FAMILY + 5); + assert!(unresolved.candidates.is_incomplete()); +} + +#[test] +fn a_weaker_rule_cannot_reinstate_a_withheld_family() { + // A token shared across a family *is* the family. Where the catalog is + // large enough that the token stays under the common-token threshold — 30 + // rows among 330 is 9% — the shared-token pass was re-offering exactly the + // rows the prefix pass had withheld, restoring the arbitrary capped slice + // the withholding exists to prevent. + let mut names = (0..30) + .map(|index| format!("Acme Branch {index:03}")) + .collect::>(); + names.extend((0..300).map(|index| format!("Unrelated Ledger {index:03}"))); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + // The scenario only exercises the path while the token stays + // discriminating: 10% of 330 is 33, and a 30-row family sits below it. + assert!( + 30 <= names.len() * COMMON_TOKEN_PERCENT / 100, + "the family would be suppressed as a common token, proving nothing" + ); + + let binding = bind_one_name(&catalog, "Acme Branch"); + let unresolved = binding.unresolved().expect("unbound"); + assert_eq!(reason(&binding), UnboundReason::NoDiscriminatingCandidate); + assert!(unresolved.candidates.listed().is_empty()); + assert_eq!(unresolved.candidates.found(), 30); + + // A decisive rule still reaches a family member on its own evidence: the + // whole key separates that one from its siblings, which is the difference + // between withholding a family and hiding a match. + let exact = bind_one_name(&catalog, "Acme Branch 017"); + assert_eq!(exact.bound_name(), Some("Acme Branch 017")); +} + +#[test] +fn a_family_within_the_bound_is_still_listed_in_full() { + let names = (0..MAX_PREFIX_FAMILY) + .map(|index| format!("ALPHAGROUP UNIT {index:02}")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let binding = bind_one_name(&catalog, "ALPHAGROUP"); + let unresolved = binding.unresolved().expect("unbound"); + assert_eq!(reason(&binding), UnboundReason::NearMiss); + assert_eq!(unresolved.candidates.listed().len(), MAX_PREFIX_FAMILY); + assert!(!unresolved.candidates.is_incomplete()); +} + +#[test] +fn the_reported_count_is_the_union_of_suppressed_and_listed_candidates() { + // A suppressed family and the candidates still worth listing are not the + // same masters. Reporting the larger of the two counts under-reports what + // the name actually reaches, and candidate_count is promised as the total + // found before truncation. + let mut names = (0..MAX_PREFIX_FAMILY + 5) + .map(|index| format!("Alpha Beta {index:02}")) + .collect::>(); + names.push("Alpha".to_string()); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + + let binding = bind_one_name(&catalog, "Alpha Beta"); + let unresolved = binding.unresolved().expect("unbound"); + // The shorter master is still listed; the family behind it is not. + assert_eq!(candidate_names(&binding), ["Alpha"]); + assert_eq!(unresolved.candidates.found(), MAX_PREFIX_FAMILY + 6); + assert!(unresolved.candidates.is_incomplete()); +} + +#[test] +fn an_identifier_hint_is_bounded_before_anything_scans_it() { + let huge = "5".repeat(MAX_NAME_CHARS + 1); + assert_eq!( + SourceEntity::with_identifier_hints(0, "Alpha Traders", [huge.as_str()]), + Err(MasterBindingError::NameTooLong) + ); + // Bounding each hint does not bound the iterator. Repeated hints fold to + // one identifier, so the deduplicated check never fired however many + // arrived, while every one of them was scanned and copied first. + let repeated = vec!["5550001001"; MAX_IDENTIFIERS_PER_NAME + 1]; + assert_eq!( + SourceEntity::with_identifier_hints(0, "Alpha Traders", repeated), + Err(MasterBindingError::TooManyIdentifiers) + ); + // The bound admits everything a usable entity could carry. + let distinct = (0..MAX_IDENTIFIERS_PER_NAME) + .map(|index| format!("555000{index:04}")) + .collect::>(); + assert!(SourceEntity::with_identifier_hints( + 0, + "Alpha Traders", + distinct.iter().map(String::as_str) + ) + .is_ok()); +} + +#[test] +fn candidate_order_is_rule_then_name_and_never_a_ranking() { + let catalog = ledgers(&[ + "ALPHA (5550000002)", + "ALPHA WHOLESALE", + "ZETA ALPHA STORE", + "ALPHA (5550000003)", + ]); + let source = SourceEntity::with_identifier_hints(0, "ALPHA", ["5550000002", "5550000003"]) + .expect("valid"); + let report = bound(&catalog, &[source]); + let unresolved = report.entities()[0].unresolved().expect("unbound"); + assert_eq!( + unresolved + .candidates + .listed() + .iter() + .map(|candidate| (candidate.catalog_name.as_str(), candidate.rule)) + .collect::>(), + [ + ("ALPHA (5550000002)", CandidateRule::SharedIdentifier), + ("ALPHA (5550000003)", CandidateRule::SharedIdentifier), + ("ALPHA WHOLESALE", CandidateRule::CatalogPrefix), + ("ZETA ALPHA STORE", CandidateRule::SharedToken), + ] + ); +} + +#[test] +fn the_report_does_not_depend_on_the_order_the_book_returned() { + let forward = ledgers(&["ALPHA SALE", "ALPHA SALES", "SALES - ALPHA", "Beta Supply"]); + let reversed = ledgers(&["Beta Supply", "SALES - ALPHA", "ALPHA SALES", "ALPHA SALE"]); + let entities = [entity("ALPHA"), entity("Beta Supply")]; + assert_eq!( + bound(&forward, &entities).entities(), + bound(&reversed, &entities).entities() + ); +} + +// --- the unbound list is the product --------------------------------------- + +#[test] +fn totals_reconcile_the_run() { + let catalog = ledgers(&["Alpha Traders", "ALPHA SALE", "ALPHA SALES"]); + let entities = [ + entity("Alpha Traders"), + entity("ALPHA"), + entity("Zeta Placeholder"), + ]; + let report = bound(&catalog, &entities); + let totals = report.totals(); + assert_eq!(totals.requested, 3); + assert_eq!(totals.bound, 1); + assert_eq!(totals.ambiguous, 1); + assert_eq!(totals.unmatched, 1); + assert_eq!(totals.requested, totals.bound + totals.unbound); + assert_eq!(totals.unbound, totals.ambiguous + totals.unmatched); + assert_eq!(report.bound().count(), totals.bound); + assert_eq!(report.unbound().count(), totals.unbound); + assert_eq!(report.class(), MasterClass::Ledger); +} + +#[test] +fn an_unbound_entry_retains_the_identity_that_will_reallocate_it() { + let catalog = ledgers(&["ALPHA (5550000001)", "BETA (5550000001)"]); + let binding = bind_one_name(&catalog, "PARTY 5550000001"); + assert_eq!( + binding.unresolved().expect("unbound").unresolved_identity, + [Identifier { + kind: IdentifierKind::Numeric, + value: "5550000001".to_string(), + }] + ); +} + +// --- the fallback is constructed, never inferred --------------------------- + +#[test] +fn an_ambiguous_entity_parks_against_a_verified_fallback() { + let catalog = ledgers(&[ + "ALPHA (5550000001)", + "BETA (5550000001)", + "Suspense Placeholder", + ]); + let report = bound(&catalog, &[entity("PARTY 5550000001")]); + let fallback = report + .assign_fallback(0, &catalog, "Suspense Placeholder") + .expect("an unbound entity may be parked"); + assert_eq!(fallback.fallback_name(), "Suspense Placeholder"); + assert_eq!(fallback.source_name(), "PARTY 5550000001"); + assert_eq!(fallback.reason(), UnboundReason::IdentifierConflict); + assert_eq!(fallback.retained_tag(), "numeric:5550000001"); + assert_eq!(fallback.class(), MasterClass::Ledger); +} + +#[test] +fn a_bound_entity_cannot_be_parked() { + let catalog = ledgers(&["Alpha Traders", "Suspense Placeholder"]); + let report = bound(&catalog, &[entity("Alpha Traders")]); + assert_eq!( + report.assign_fallback(0, &catalog, "Suspense Placeholder"), + Err(MasterBindingError::FallbackNotInCatalog) + ); +} + +#[test] +fn a_fallback_master_that_does_not_exist_is_refused() { + // A suspense ledger that was never created is how one batch was lost. + let catalog = ledgers(&["Alpha Traders", "Beta Supply"]); + let report = bound(&catalog, &[entity("Zeta Placeholder")]); + assert_eq!( + report.assign_fallback(0, &catalog, "Suspense Placeholder"), + Err(MasterBindingError::FallbackNotInCatalog) + ); +} + +#[test] +fn a_fallback_cannot_be_drawn_from_another_catalog_class_or_another_report() { + // A stock-item binding parked against a ledger catalog was a representable + // state that nothing downstream could detect. + let stock = MasterCatalog::new(MasterClass::StockItem, ["PH-01A-B00", "Scrap Placeholder"]) + .expect("valid"); + let ledger = ledgers(&["Alpha Traders", "Suspense Placeholder"]); + let stock_report = bound(&stock, &[entity("Zeta Placeholder")]); + assert_eq!( + stock_report.assign_fallback(0, &ledger, "Suspense Placeholder"), + Err(MasterBindingError::ClassMismatch) + ); + // An index outside this report cannot name another report's entity. + assert_eq!( + stock_report.assign_fallback(7, &stock, "Scrap Placeholder"), + Err(MasterBindingError::ClassMismatch) + ); + // Nor may a *same-class* catalog the report was never produced from supply + // the fallback: class is not provenance, and the master would never have + // been a candidate for this entity. + let other_ledgers = ledgers(&["Alpha Traders", "Different Suspense"]); + let ledger_report = bound(&ledger, &[entity("Zeta Placeholder")]); + assert_eq!( + ledger_report.assign_fallback(0, &other_ledgers, "Different Suspense"), + Err(MasterBindingError::ClassMismatch) + ); + assert_ne!(ledger.fingerprint(), other_ledgers.fingerprint()); + // The same masters read twice fingerprint alike, whatever order they came + // back in — a re-read must not invalidate a report. + let reordered = ledgers(&["Suspense Placeholder", "Alpha Traders", "Beta Supply"]); + let forward = ledgers(&["Alpha Traders", "Beta Supply", "Suspense Placeholder"]); + assert_eq!(reordered.fingerprint(), forward.fingerprint()); + assert_eq!( + MasterBindingError::ClassMismatch.safe_reason_code(), + "master_class_mismatch" + ); +} + +#[test] +fn the_adr_quotes_the_thresholds_this_module_actually_uses() { + // ADR 0016 is the contract two surfaces integrate against, so a threshold + // that moves in code and not in the document sends a future integration + // the wrong rule. "Remember to update the record" is the kind of rule this + // project prefers to replace with something that fails. + const ADR: &str = include_str!("../../../../docs/adr/0016-master-binding-authority.md"); + for (constant, value) in [ + ( + "MIN_NUMERIC_IDENTIFIER_DIGITS", + MIN_NUMERIC_IDENTIFIER_DIGITS, + ), + ("MIN_CODE_IDENTIFIER_DIGITS", MIN_CODE_IDENTIFIER_DIGITS), + ("MIN_CODE_IDENTIFIER_CHARS", MIN_CODE_IDENTIFIER_CHARS), + ("MAX_CANDIDATES_PER_ENTITY", MAX_CANDIDATES_PER_ENTITY), + ("COMMON_TOKEN_PERCENT", COMMON_TOKEN_PERCENT), + ] { + // A percentage reads naturally as `(10%)`; both spellings count, and + // neither lets a changed number pass. + let plain = format!("`{constant}` ({value})"); + let percent = format!("`{constant}` ({value}%)"); + assert!( + ADR.contains(&plain) || ADR.contains(&percent), + "ADR 0016 does not quote {constant} as {value}; it must read {plain:?}" + ); + } +} + +// --- the vocabulary is stable ---------------------------------------------- + +#[test] +fn reason_and_error_codes_are_stable_and_safe() { + assert_eq!( + UnboundReason::IdentifierConflict.safe_reason_code(), + "master_binding_identifier_conflict" + ); + assert_eq!( + UnboundReason::IdentifierNameConflict.safe_reason_code(), + "master_binding_identifier_name_conflict" + ); + assert_eq!( + UnboundReason::NameAmbiguous.safe_reason_code(), + "master_binding_name_ambiguous" + ); + assert_eq!( + UnboundReason::NearMiss.safe_reason_code(), + "master_binding_near_miss" + ); + assert_eq!( + MasterBindingError::CatalogEmpty.safe_reason_code(), + "master_catalog_empty" + ); +} + +#[test] +fn every_unresolved_shape_survives_serialization() { + // A newtype variant under internal tagging cannot carry a sequence, and it + // failed at runtime on the *most common* unresolved result while the other + // three variants serialized fine. No test caught it because none had ever + // serialized an `Unresolved` — only a `Bound`. + let listed = ledgers(&["ALPHA SALE", "ALPHA SALES", "SALES - ALPHA", "Beta Supply"]); + let family = (0..MAX_PREFIX_FAMILY + 5) + .map(|index| format!("ALPHAGROUP UNIT {index:02}")) + .collect::>(); + let family = MasterCatalog::new(MasterClass::Ledger, &family).expect("valid"); + let missing = ledgers(&["Alpha Traders", "Beta Supply"]); + + for (label, binding) in [ + ("listed", bind_one_name(&listed, "ALPHA")), + ("withheld", bind_one_name(&family, "ALPHAGROUP")), + ("none", bind_one_name(&missing, "Zeta Placeholder")), + ] { + let json = serde_json::to_string(&binding) + .unwrap_or_else(|error| panic!("{label} failed to serialize: {error}")); + let back: EntityBinding = serde_json::from_str(&json) + .unwrap_or_else(|error| panic!("{label} failed to deserialize: {error}")); + assert_eq!(back, binding, "{label} did not round-trip"); + } +} + +#[test] +fn a_bound_status_serializes_without_a_score_field() { + let catalog = ledgers(&["Alpha Traders"]); + let binding = bind_one_name(&catalog, "Alpha Traders"); + let json = serde_json::to_value(&binding).expect("serializable"); + assert_eq!(json["status"], "bound"); + assert_eq!(json["catalog_name"], "Alpha Traders"); + assert_eq!(json["basis"], "exact_name"); + assert!(json.get("score").is_none()); + assert!(json.get("confidence").is_none()); +} + +// --------------------------------------------------------------------------- +// Characterization against a realistically shaped book +// +// Every rule above is tested in isolation on a handful of names. Three of the +// rules only engage at scale — common-token suppression needs 20+ entries, +// candidate capping needs 25+, and the prefix ranges only matter when many +// keys share a head — so their interaction is untested by any of it. +// +// This section fabricates one 200-master catalog carrying the naming +// pathologies actually recorded (a firm word on most ledgers, numbers typed +// into party names, a near-duplicate sales trio, a masked bank last-four) and +// pins the *outcome* for a document-sized set of source names. +// +// The assertion that matters is not the count. It is that **no entity binds to +// a master a human would not have chosen**: a wrong bind puts money against the +// wrong party, and is strictly worse than an unbound row. The counts are pinned +// underneath it so that loosening a threshold has to move a number in a diff. +// +// This is fabricated input. It characterizes the rules and is not evidence +// about any Tally instance or any real book's bindability. +// --------------------------------------------------------------------------- + +const GREEK: [&str; 20] = [ + "ALPHA", "BETA", "GAMMA", "DELTA", "EPSILON", "ZETA", "ETA", "THETA", "IOTA", "KAPPA", + "LAMBDA", "MU", "NU", "XI", "OMICRON", "PI", "RHO", "SIGMA", "TAU", "UPSILON", +]; + +/// One fabricated book: 200 ledgers, shaped like a small trading firm's. +fn fabricated_book() -> Vec { + let mut names = Vec::new(); + // 20 party ledgers with a number typed into the name, as operators do. + for (index, greek) in GREEK.iter().enumerate() { + names.push(format!( + "{greek} PLACEHOLDER ({})", + 5_550_001_001_u64 + index as u64 + )); + } + // 20 party ledgers without one. + for greek in GREEK { + names.push(format!("{greek} PLACEHOLDER TRADING CO")); + } + // The near-duplicate trio that one engagement actually met. + names.push("ALPHA SALE".to_string()); + names.push("ALPHA SALES".to_string()); + names.push("SALES - ALPHA".to_string()); + // Tax heads, which share heavy word overlap with each other. + for head in ["CGST", "SGST", "IGST"] { + for side in ["OUTPUT", "INPUT"] { + for rate in ["9%", "18%"] { + names.push(format!("{head} {side} {rate}")); + } + } + } + // Banks, carrying a masked last-four rather than a full account number. + names.push("PLACEHOLDER BANK CA 2129".to_string()); + names.push("PLACEHOLDER BANK OD 7745".to_string()); + // The accounts every book has. + for name in [ + "Cash", + "Suspense Placeholder", + "Round Off", + "Profit & Loss A/c", + ] { + names.push(name.to_string()); + } + // Filler carrying one firm-wide word, to the size of a real small book. + let mut index = 0; + while names.len() < 200 { + names.push(format!("PLACEHOLDER UNIT {index:03}")); + index += 1; + } + names +} + +#[derive(Debug, PartialEq, Eq)] +enum Expected { + /// The master a human reading the source would have chosen. + Bound(&'static str), + Unbound(UnboundReason), +} + +/// What one document names, and what a human would do with each. +fn fabricated_document() -> Vec<(&'static str, Option<&'static str>, Expected)> { + vec![ + // Named exactly as the book spells it. + ("Cash", None, Expected::Bound("Cash")), + ("CGST OUTPUT 9%", None, Expected::Bound("CGST OUTPUT 9%")), + // Case noise alone resolves, and so does spacing noise: §9.4d measured + // leading whitespace and a collapsed run matching on licensed 7.1. + ("cgst output 9%", None, Expected::Bound("CGST OUTPUT 9%")), + ( + " cgst output 9% ", + None, + Expected::Bound("CGST OUTPUT 9%"), + ), + ( + "beta placeholder trading co", + None, + Expected::Bound("BETA PLACEHOLDER TRADING CO"), + ), + // The engagement case: the source names the party its own way and + // carries the number in a separate column. Name matching would offer + // twenty wrong parties; the number decides. + ( + "GAMMA. K.", + Some("5550001003"), + Expected::Bound("GAMMA PLACEHOLDER (5550001003)"), + ), + // Same, with the number inside the name rather than a hint. + ( + "DELTA K 5550001004", + None, + Expected::Bound("DELTA PLACEHOLDER (5550001004)"), + ), + // A truncated party name: one candidate, and one candidate is still + // not a decision. + ( + "EPSILON PLACEHOLDER TRADING", + None, + Expected::Unbound(UnboundReason::NearMiss), + ), + // The near-duplicate trio. Nothing here may resolve. + ("ALPHA SALE", None, Expected::Bound("ALPHA SALE")), + ("ALPHA", None, Expected::Unbound(UnboundReason::NearMiss)), + // A masked bank last-four must not bind on four digits. + ( + "PLACEHOLDER BANK 2129", + None, + Expected::Unbound(UnboundReason::NearMiss), + ), + // A party the book simply does not have. + ( + "OMEGA WHOLESALE", + None, + Expected::Unbound(UnboundReason::NoCandidate), + ), + // A number the book does not carry: the hint finds nothing, and the + // name is left to answer on its own. + ( + "PSI SUPPLY", + Some("5559999999"), + Expected::Unbound(UnboundReason::NoCandidate), + ), + ] +} + +#[test] +fn a_document_against_a_realistic_book_binds_only_where_a_human_would() { + let names = fabricated_book(); + assert_eq!(names.len(), 200); + let catalog = + MasterCatalog::new(MasterClass::Ledger, &names).expect("the fabricated book is valid"); + assert_eq!(catalog.master_count(), 200); + + let document = fabricated_document(); + let entities = document + .iter() + .enumerate() + .map(|(position, (name, hint, _))| match hint { + Some(hint) => SourceEntity::with_identifier_hints(position, name, [*hint]), + None => SourceEntity::new(position, name), + }) + .map(|entity| entity.expect("fabricated source names are valid")) + .collect::>(); + let report = bound(&catalog, &entities); + + for ((source, _, expected), binding) in document.iter().zip(report.entities()) { + match (&binding.status, expected) { + (BindingStatus::Bound { catalog_name, .. }, Expected::Bound(intended)) => assert_eq!( + catalog_name, intended, + "{source:?} bound to a master a human would not have chosen" + ), + ( + BindingStatus::Ambiguous(unresolved) | BindingStatus::Unmatched(unresolved), + Expected::Unbound(intended), + ) => assert_eq!( + unresolved.reason, *intended, + "{source:?} was unbound for an unintended reason" + ), + (status, expected) => { + panic!("{source:?}: expected {expected:?}, got {status:?}") + } + } + } + + // The shape of the answer, pinned so a loosened threshold moves a number. + let totals = report.totals(); + assert_eq!(totals.requested, 13); + assert_eq!(totals.bound, 8); + assert_eq!(totals.ambiguous, 3); + assert_eq!(totals.unmatched, 2); + assert_eq!(totals.requested, totals.bound + totals.unbound); +} + +#[test] +fn a_firm_wide_word_does_not_drag_the_whole_book_into_every_candidate_list() { + // "placeholder" is carried by most of this book. Without suppression the + // unbound list stops being a work item and becomes a second data-entry job. + let names = fabricated_book(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let carriers = names + .iter() + .filter(|name| name.to_lowercase().contains("placeholder")) + .count(); + assert!(carriers > 100, "the fabricated book must exercise this"); + + let binding = bind_one_name(&catalog, "OMEGA PLACEHOLDER"); + let unresolved = binding.unresolved().expect("unbound"); + assert!( + unresolved.candidates.found() <= MAX_CANDIDATES_PER_ENTITY, + "a firm-wide word pulled in {} candidates", + unresolved.candidates.found() + ); +} + +/// The mutations a source document actually applies to a name it copied from +/// somewhere else: case, spacing, a dropped tail, a dropped last word. +fn source_mutations(name: &str) -> Vec { + let mut mutations = vec![name.to_uppercase(), name.to_lowercase()]; + mutations.push(format!( + " {} ", + name.split_whitespace().collect::>().join(" ") + )); + let characters = name.chars().collect::>(); + let kept = characters.len() * 4 / 5; + if kept >= MIN_PREFIX_KEY_CHARS { + mutations.push(characters[..kept].iter().collect()); + } + let words = name.split_whitespace().collect::>(); + if words.len() > 2 { + mutations.push(words[..words.len() - 1].join(" ")); + } + mutations +} + +#[test] +fn no_mutation_of_a_master_name_ever_binds_to_a_different_master() { + // The safety property, stated over a whole book rather than three chosen + // names: a wrong bind puts money against the wrong party, and is strictly + // worse than an unbound row. Roughly a thousand cases. + // + // Mutations that collide with *another* master under the comparison key + // are excluded, and deliberately so: truncating `ALPHA SALES` by one + // character yields `ALPHA SALE`, which is a real and different ledger. No + // rule can distinguish a truncation of one name from an exact spelling of + // another, and binding it to the name it actually spells is correct. + // + // **This sweep was checked against two positive controls**, because an + // assertion that has never failed is not yet known to be an instrument: + // + // - Resolving a near-miss to its first-ordered candidate — precisely what + // the deleted MCP helper did through `exact_live_spelling` — trips it on + // a truncated party name. So it does report presence. + // - Binding a *lone* candidate does **not** trip it, because in this book a + // lone candidate is nearly always the master the mutation came from. That + // regression is caught by `a_single_candidate_still_does_not_bind` and by + // the two prefix tests instead. + // + // Read this test as "no mutation reaches the wrong master", never as "no + // rule change can loosen binding". + let names = fabricated_book(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let keys = names + .iter() + .map(|name| (comparison_key(name), name.as_str())) + .collect::>(); + + // What the *wide* fold would resolve. Narrowing the resolving fold to the + // three transformations §9.4b verified is only defensible if it withdraws + // answers, never masters, so the cases it used to settle are tracked by + // name rather than by a percentage that drifts with the fixture. + let wide = names + .iter() + .map(|name| (master_identity_key(name), name.as_str())) + .collect::>(); + + let mut checked = 0_usize; + let mut self_bound = 0_usize; + let mut self_offered = 0_usize; + for name in &names { + for mutation in source_mutations(name) { + let key = comparison_key(&mutation); + if keys.get(&key).is_some_and(|owner| owner != name) { + continue; // the mutation spells a different real ledger + } + checked += 1; + let binding = bind_one_name(&catalog, &mutation); + let wide_would_bind = wide.get(&master_identity_key(&mutation)) == Some(&name.as_str()); + match binding.bound_name() { + None => { + let offered = candidate_names(&binding).iter().any(|shown| shown == name); + if offered { + self_offered += 1; + } + // The invariant that makes the narrowing a trade rather than + // a loss: anything the wide fold settled is still shown. + assert!( + !wide_would_bind || offered, + "narrowing the fold hid {name:?} from its own mutation {mutation:?}" + ); + } + Some(bound_to) => { + assert_eq!( + bound_to, name, + "mutation {mutation:?} of {name:?} bound to a different master" + ); + self_bound += 1; + } + } + } + } + assert!( + checked > 900, + "the sweep must actually cover the book: {checked}" + ); + // Most of this book's mutations are spacing and separator noise, and §9.4d + // measured Tally folding all of it on the SKU this writes to — so most of + // them resolve again. The per-case invariant above still holds and is the + // point: anything the wide fold reaches is bound or shown, never hidden. + // + // On this book the two folds now agree on every mutation, so the sweep does + // **not** exercise the gap between them. The cases that still separate them + // — an en dash, an underscore, an NFD spelling — are covered by + // `space_hyphen_and_slash_are_one_separator_in_both_directions` and + // `two_encodings_of_one_name_are_two_masters_to_tally_and_so_to_this` + // instead, and this comment exists so nobody reads a green sweep as + // evidence about them. + assert!( + self_bound * 2 > checked, + "most mutations should resolve once the fold matches the gateway: \ + {self_bound} bound of {checked}" + ); + assert!( + self_bound + self_offered > checked * 2 / 3, + "{self_bound} bound and {self_offered} offered of {checked}" + ); +} + +#[test] +fn a_number_typed_into_a_master_name_finds_it_from_any_source_name() { + // The rule that decided the case fuzzy matching got wrong, exercised + // against every party ledger in the book rather than one. + let names = fabricated_book(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + let numbered = names + .iter() + .filter(|name| name.contains("PLACEHOLDER (")) + .collect::>(); + assert_eq!(numbered.len(), 20); + + for name in numbered { + let number = name + .rsplit_once('(') + .and_then(|(_, tail)| tail.strip_suffix(')')) + .expect("fabricated party names carry a number"); + // A source name sharing nothing with the ledger name at all. + let entity = SourceEntity::with_identifier_hints(0, "UNRELATED SOURCE PARTY", [number]) + .expect("valid"); + let report = bound(&catalog, &[entity]); + assert_eq!( + report.entities()[0].bound_name(), + Some(name.as_str()), + "the number typed into {name:?} did not find it" + ); + } +} + +#[test] +fn a_dash_variant_does_not_spell_a_short_code_into_a_decisive_one() { + // The code threshold was `canonical.len()`, which is UTF-8 bytes. An en + // dash is three of them, so `AB–123` measured eight and cleared a bound + // meant for eight *characters* while carrying five alphanumerics. Its + // ASCII twin reduces to `AB123` and is refused, so the same code decided + // or did not on the strength of which dash the document happened to use — + // and the wrong master was reached only in the spelling nobody checks. + let catalog = ledgers(&["Sales AB\u{2013}123", "Beta Supply"]); + let binding = bind_one_name(&catalog, "Purchases AB\u{2013}123"); + assert_eq!( + binding.bound_name(), + None, + "five alphanumerics identifier-bound because one of them was three bytes" + ); + + // Padding is the same defect without the twin to compare against: bytes + // counted the dashes, characters would have counted them too. + let padded = ledgers(&["Sales AB\u{2013}\u{2013}\u{2013}123", "Beta Supply"]); + assert_eq!( + bind_one_name(&padded, "Purchases AB\u{2013}\u{2013}\u{2013}123").bound_name(), + None, + "a code padded with dashes cleared the threshold on its punctuation" + ); + + // The measured shape still binds: `PH-01-AB-00` is eight alphanumerics + // once the separators §9.4d measured are discarded, and it identifies. + let stock = ledgers(&["Sales PH-01-AB-00", "Beta Supply"]); + assert_eq!( + bind_one_name(&stock, "Purchases PH01AB00").bound_name(), + Some("Sales PH-01-AB-00"), + "the fold §9.4d measured stopped working" + ); +} + +#[test] +fn a_withheld_family_counts_the_masters_the_other_identifier_listed_too() { + // The count was the largest skipped family's size, which ignores every + // master the entity's *other* identifiers reached. Thirty masters share + // one number and a thirty-first carries the other: the operator was told + // thirty, and the master that made the two disagree was not in the number. + let mut names = (0..MAX_CANDIDATES_PER_ENTITY + 5) + .map(|index| format!("Shared Party {index:03} (5550007777)")) + .collect::>(); + names.push("Lone Party (5550008888)".to_string()); + let family = MAX_CANDIDATES_PER_ENTITY + 5; + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + + let entity = + SourceEntity::with_identifier_hints(0, "Zeta Holdings", ["5550007777", "5550008888"]) + .expect("valid"); + let binding = bound(&catalog, &[entity]) + .entities() + .first() + .cloned() + .expect("one entity in, one binding out"); + + assert_eq!(reason(&binding), UnboundReason::IdentifierConflict); + let unresolved = binding.unresolved().expect("unbound"); + assert!( + unresolved.candidates.is_incomplete(), + "the family was listed" + ); + assert_eq!( + unresolved.candidates.found(), + family + 1, + "the master the other identifier listed was not counted" + ); + // Still without building the union: the large family is a membership test, + // never an expansion. + assert!( + family + 1 > MAX_CANDIDATES_PER_ENTITY, + "this fixture no longer exercises the skip" + ); +} + +#[test] +fn hint_variants_of_one_name_do_not_crowd_out_a_key_that_repeats() { + // The memo is keyed by the source key *and* the masters the identifiers + // reached, but repetition was counted on the key alone. Every hint variant + // of one name therefore counted as repeated, filled the memo with entries + // nothing asks for twice, and the pair that genuinely repeated behind them + // could no longer be inserted — the same stall as the source-order defect, + // through a different door. + let names = (0..60) + .map(|index| format!("Acme Branch {index:05}")) + .collect::>(); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + + // Distinct hints, one name: same key, different memo key, each asked once. + let mut entities = (0..MAX_CANDIDATE_MEMO_ENTRIES) + .map(|index| { + SourceEntity::with_identifier_hints( + index, + "Acme Branch", + [format!("5550{index:06}").as_str()], + ) + .expect("valid") + }) + .collect::>(); + // And then a pair that does repeat, four times over. + entities.extend((0..4).map(|offset| { + SourceEntity::with_identifier_hints( + MAX_CANDIDATE_MEMO_ENTRIES + offset, + "Acme Branch", + ["5559999999"], + ) + .expect("valid") + })); + + super::CANDIDATE_SEARCHES.with(|count| count.set(0)); + let report = bound(&catalog, &entities); + let searches = super::CANDIDATE_SEARCHES.with(std::cell::Cell::get); + assert_eq!(report.totals().requested, MAX_CANDIDATE_MEMO_ENTRIES + 4); + assert!( + searches <= MAX_CANDIDATE_MEMO_ENTRIES + 1, + "the repeated pair was searched more than once behind {MAX_CANDIDATE_MEMO_ENTRIES} variants: {searches} searches" + ); +} + +#[test] +fn a_request_within_every_other_bound_is_still_refused_on_its_total_size() { + // The count bound and the per-name bound do not bound their product. Each + // of these names is valid, and the list is well inside the entity count; + // together they are more than the aggregate the catalog side has always + // had, and this module would have cloned them into a report first. + // One long token rather than many short ones: the aggregate is what this + // test is about, and a name of 1,800 words would spend the whole test in + // identifier extraction proving nothing extra. + let name = format!("Zeta{}", "z".repeat(MAX_NAME_CHARS - 5)); + assert!( + name.chars().count() < MAX_NAME_CHARS, + "each name must stay individually valid" + ); + let catalog = ledgers(&["Alpha Supply", "Beta Supply"]); + let entities = (0..1_200) + .map(|index| SourceEntity::new(index, &name).expect("each name is valid alone")) + .collect::>(); + assert!( + entities.len() < MAX_SOURCE_ENTITIES, + "the count bound must not be what refuses this" + ); + assert_eq!( + bind(&catalog, &entities), + Err(MasterBindingError::SourceNamesTooLarge) + ); + + // And the bound does not refuse an ordinary request: the same names, well + // under the aggregate, bind as before. + let ordinary = (0..40) + .map(|index| SourceEntity::new(index, &name).expect("valid")) + .collect::>(); + assert!(bind(&catalog, &ordinary).is_ok()); +} + +#[test] +fn a_date_range_fused_by_its_own_separator_is_still_dates() { + // The separator works in both directions. Removing `-` reveals a date in + // `2025-09-11`, and hides two in `DATED20250911-20250912`: the canonical + // carries one sixteen-digit run that reads as no date, and `is_period` does + // not see it either, because its eight-digit case admits a year followed by + // a year and `0911` is neither. A period label is the single thing two + // unrelated masters most reliably share, so identifying on one is the + // wrong-party bind this guard exists to prevent. + let catalog = ledgers(&["Sales DATED20250911-20250912", "Beta Supply"]); + assert_eq!( + bind_one_name(&catalog, "Purchases DATED20250911-20250912").bound_name(), + None, + "a fused date range identified two unrelated masters with each other" + ); + + // The direction the canonical check exists for still holds: a date written + // with separators is a date once they are removed. + let punctuated = ledgers(&["Sales 2025-09-11", "Beta Supply"]); + assert_eq!( + bind_one_name(&punctuated, "Purchases 2025-09-11").bound_name(), + None, + "a punctuated date identified" + ); + + // And a genuine code still identifies, so the guard has not swallowed the + // rule it guards. + let coded = ledgers(&["Sales AB-123456", "Beta Supply"]); + assert_eq!( + bind_one_name(&coded, "Purchases AB-123456").bound_name(), + Some("Sales AB-123456"), + "the identifier rule stopped working" + ); +} + +#[test] +fn the_expensive_repeated_search_is_the_one_the_memo_keeps() { + // `worth_holding` refused to cache a result larger than the candidate cap, + // which read as prudence and excluded exactly the search the memo exists + // for: a name reaching a large family through shared tokens re-ran it once + // per row. The search now returns a capped list and the full count, so + // every result is small enough to hold. + // Forty masters carry one distinctive token, in a catalog of four hundred. + // Forty is exactly the common-token limit rather than over it, so the token + // still discriminates and the search returns all forty — more than the + // candidate cap, which is what made the old memo refuse to hold it. The + // token sits at the end of each name so this reaches the shared-token rule + // rather than the prefix family, which withholds instead of listing. + let mut names = (0..360) + .map(|index| format!("Alpha Placeholder {index:04}")) + .collect::>(); + names.extend((0..40).map(|index| format!("Beta Placeholder {index:04} Zetaomega"))); + let catalog = MasterCatalog::new(MasterClass::Ledger, &names).expect("valid"); + + let entities = (0..64) + .map(|index| SourceEntity::new(index, "Zetaomega").expect("valid")) + .collect::>(); + + super::CANDIDATE_SEARCHES.with(|count| count.set(0)); + let report = bound(&catalog, &entities); + let searches = super::CANDIDATE_SEARCHES.with(std::cell::Cell::get); + assert_eq!(report.totals().requested, 64); + assert_eq!( + searches, 1, + "a repeated name reaching a large family was searched {searches} times" + ); + + // The count the operator sees is the whole union, not the capped listing. + let unresolved = report.entities()[0].unresolved().expect("unbound"); + assert!( + unresolved.candidates.found() > MAX_CANDIDATES_PER_ENTITY, + "this fixture no longer exercises a result larger than the cap" + ); + assert!(unresolved.candidates.listed().len() <= MAX_CANDIDATES_PER_ENTITY); +} + +#[test] +fn an_ambiguity_lists_every_master_that_caused_it() { + // The reason is decided on the resolving fold and the candidates were + // gathered from the wide one, which is not always coarser: the wide fold + // replaces `-` but not `/`, so `AB/CD` and `AB CD` are one master to + // `verified_fold` and two to `master_identity_key`. Both tokens are below + // the shared-token threshold, so nothing else restored the space spelling, + // and the operator was shown an ambiguity with a complete-looking list of + // one — asked to choose between masters they could not see. + let catalog = ledgers(&["AB/CD", "AB CD", "Beta Supply"]); + let binding = bind_one_name(&catalog, "ab/cd"); + assert_eq!(reason(&binding), UnboundReason::NameAmbiguous); + assert_eq!(binding.bound_name(), None); + assert_eq!(candidate_names(&binding), ["AB CD", "AB/CD"]); + assert_eq!( + binding.unresolved().expect("unbound").candidates.found(), + 2, + "the count agreed with the short list rather than with the collision" + ); +} diff --git a/src-tauri/src/agent_catalog.rs b/src-tauri/src/agent_catalog.rs index b1754c8d8..b3dffa5c5 100644 --- a/src-tauri/src/agent_catalog.rs +++ b/src-tauri/src/agent_catalog.rs @@ -148,7 +148,7 @@ pub(super) fn registered_tool_definitions(import_enabled: bool, writes_enabled: json!({"type":"object", "additionalProperties":false}), ), "validate_masters" => ( - "Validate 1–100 nonblank ledger names (at most 1024 characters each). Near-miss suggestions are bounded to 25 names and 8192 UTF-8 bytes per requested name, with total count and truncation reported.", + "Bind 1–100 nonblank ledger names (at most 1024 characters each) against the live catalogue. An identifier embedded in a master name is matched before the name itself. `match_state` is exact, normalized, identifier, near_miss or missing; only `exact` is admitted by build_import_xml, and a bound row alone carries `exact_live_spelling`. A near-miss is never resolved: it returns candidates with the rule that surfaced each, bounded to 25 names and 8192 UTF-8 bytes per requested name, with total count and truncation reported. There is no ranking and no score.", json!({"type":"object", "additionalProperties":false, "required":["company_guid","ledgers"], "properties":{"company_guid":{"type":"string"},"ledgers":{"type":"array","minItems":1,"maxItems":agent_import::MAX_MASTER_NAMES,"items":{"type":"string","minLength":1,"maxLength":agent_import::MAX_MASTER_NAME_CHARS,"pattern":r"\S"}}}}), ), "build_import_xml" => ( diff --git a/src-tauri/src/agent_failure_tests.rs b/src-tauri/src/agent_failure_tests.rs index 182e4cc3a..ae933e080 100644 --- a/src-tauri/src/agent_failure_tests.rs +++ b/src-tauri/src/agent_failure_tests.rs @@ -134,3 +134,47 @@ async fn import_post_read_failures_retain_source_evidence_and_admission_errors_s assert!(!directory.path().join("imports").exists()); } } + +#[tokio::test] +async fn a_name_the_core_refuses_costs_no_live_read() { + // The tool schema and the binding core do not admit the same names: a + // control character satisfies the schema's length and count rules and is + // refused by `SourceEntity::new`. Parsing after the company verification + // and the catalogue read meant the refusal arrived only after two live + // round trips against the operator's books, and after evidence of them had + // been retained — for a request that could not have succeeded against any + // catalogue. + // + // The control is the endpoint itself: nothing is listening on it. A reach + // for Tally before the request is parsed cannot come back as + // `master_name_unsafe`, it comes back as a failure to connect — so the + // reason code is the proof that no read was attempted, not a restatement + // of the assertion. + let directory = tempfile::tempdir().unwrap(); + let server = Server::new(Settings { + endpoint: TallyEndpointConfig { + host: "127.0.0.1".into(), + // Port 1 is privileged and unbound here; any connection is refused. + port: 1, + }, + data_dir: directory.path().to_path_buf(), + max_rows: 10, + max_bytes: 200_000, + redaction: Redaction::None, + import_enabled: true, + writes_enabled: false, + }); + + let refused = server + .call_tool_response( + "validate_masters", + json!({"company_guid": CAPTURED_GUID, "ledgers": ["Cash\u{7}"]}), + ) + .await; + assert_eq!(refused.value["isError"], true); + let content: Value = + serde_json::from_str(refused.value["content"][0]["text"].as_str().unwrap()).unwrap(); + assert_eq!(content["result"]["error"]["code"], "master_name_unsafe"); + // Nothing was read, so there is nothing to commit to. + assert_eq!(refused.value["structuredContent"]["evidence"]["bytes"], 0); +} diff --git a/src-tauri/src/agent_import.rs b/src-tauri/src/agent_import.rs index 23adc1865..5549eae4b 100644 --- a/src-tauri/src/agent_import.rs +++ b/src-tauri/src/agent_import.rs @@ -8,6 +8,10 @@ use crate::tally::standard_ledger_catalog::{ admit_standard_ledger_catalog_request, parse_standard_ledger_catalog_response, render_standard_ledger_catalog_request, }; +use bridge_tally_core::master_binding::{ + self, BindingBasis, BindingStatus, Candidates, EntityBinding, MasterCatalog, MasterClass, + SourceEntity, +}; use bridge_tally_core::ExactDecimal; use bridge_tally_protocol::native_outstandings::{ parse_native_group_snapshot, render_native_group_snapshot_request, @@ -45,7 +49,6 @@ mod persistence; #[path = "agent_import_post.rs"] mod post; use std::path::{Path, PathBuf}; -use unicode_normalization::UnicodeNormalization; use uuid::Uuid; struct ImportProfileObservation { @@ -342,15 +345,28 @@ impl Server { .collect::>>() .filter(|names| !names.is_empty()) .ok_or_else(|| "ledgers_required".to_string())?; + // Before either read. A name the core refuses is refused at any + // catalogue, so verifying the company and reading its ledgers first + // would spend two live round trips — and retain evidence of them — to + // reach a failure that was decidable from the request alone. + let entities = + source_entities(&ledgers.into_iter().map(str::to_string).collect::>()) + .map_err(ToolFailure::from)?; let (company, identity, identity_evidence) = self.verified_company(guid).await?; let (catalogue, evidence) = self .read_ledger_catalogue(&identity, &company.name) .await .map_err(|failure| failure.with_prior_evidence(identity_evidence.clone()))?; - let report = ledgers - .into_iter() - .map(|wanted| master_match(wanted, &catalogue)) - .collect::>(); + let report = master_report(&entities, &catalogue) + // The catalogue read already succeeded, so its request/response + // commitments belong in the failure too; attaching identity evidence + // alone would omit a Tally read that actually happened. + .map_err(|code| { + ToolFailure::from(code).with_prior_evidence(combine_evidence( + identity_evidence.clone(), + evidence.clone(), + )) + })?; let hash = sha256_json(&catalogue); Ok(ToolOutcome { payload: json!({"company": company_json(&company, std::slice::from_ref(&company)), "result": {"masters": report, "catalogue_evidence_sha256": hash}}), @@ -433,7 +449,7 @@ impl Server { .await .map(|(names, catalogue, _, evidence)| (names, catalogue, evidence))?; accumulated = combine_evidence(accumulated.clone(), catalogue_evidence.clone()); - let report = masters_for_payload(&payload, &catalogue); + let report = masters_for_payload(&payload, &catalogue)?; if report.iter().any(|value| value["match_state"] != "exact") { return Ok(ToolOutcome { payload: json!({"company": company_json(&company, std::slice::from_ref(&company)), "result": { @@ -1559,11 +1575,31 @@ fn totals(vouchers: &[ImportVoucher]) -> Result<(ExactDecimal, ExactDecimal), St Ok((debit, credit)) } -fn masters_for_payload(payload: &ImportPayload, catalogue: &[String]) -> Vec { - requested_ledger_names(payload) - .into_iter() - .map(|name| master_match(&name, catalogue)) - .collect() +fn masters_for_payload( + payload: &ImportPayload, + catalogue: &[String], +) -> Result, String> { + master_report( + &source_entities(&requested_ledger_names(payload))?, + catalogue, + ) +} + +/// Parses requested names into source entities, which is where the core's own +/// bounds are enforced — a control character, or more identifiers than one name +/// may carry. +/// +/// Kept separate from `master_report` so a caller can run it **before** it +/// reads Tally. A request the core will refuse cannot succeed at any catalogue, +/// so spending a live read and collecting evidence of it first buys nothing and +/// costs an external round trip against the operator's books. +fn source_entities(requested: &[String]) -> Result, String> { + requested + .iter() + .enumerate() + .map(|(position, name)| SourceEntity::new(position, name)) + .collect::, _>>() + .map_err(|error| error.safe_reason_code().to_string()) } fn requested_ledger_names(payload: &ImportPayload) -> Vec { @@ -1577,53 +1613,101 @@ fn requested_ledger_names(payload: &ImportPayload) -> Vec { .collect() } -fn master_match(wanted: &str, catalogue: &[String]) -> Value { - if catalogue.iter().any(|name| name == wanted) { - return json!({"requested": party_name(wanted), "match_state":"exact", "exact_live_spelling":party_name(wanted)}); - } - let key = master_key(wanted); - let candidates = catalogue - .iter() - .filter(|name| { - let candidate = master_key(name); - candidate == key || candidate.starts_with(&key) || key.starts_with(&candidate) - }) - .collect::>(); - let candidate_count = candidates.len(); - if candidate_count == 0 { - json!({"requested":party_name(wanted),"match_state":"missing"}) - } else { - let mut bytes = 0_usize; - let candidates = candidates - .into_iter() - .take(25) - .take_while(|name| { - bytes = bytes.saturating_add(name.len()); - bytes <= 8192 +/// Bounds the candidate names one unbound entity may copy into a tool result. +/// The binding contract bounds the candidate *count*; this bounds their bytes, +/// which is an egress concern rather than a matching one. +const MAX_CANDIDATE_RESULT_BYTES: usize = 8_192; + +/// Binds requested ledger names against the observed catalogue. +/// +/// The rules live in `bridge_tally_core::master_binding` so this tool and the +/// desktop preparation screen cannot drift apart; see +/// `docs/adr/0016-master-binding-authority.md`. This function only renders the +/// report, and it never promotes a candidate into a spelling. +fn master_report(entities: &[SourceEntity], catalogue: &[String]) -> Result, String> { + let catalog = MasterCatalog::new(MasterClass::Ledger, catalogue) + .map_err(|error| error.safe_reason_code().to_string())?; + let report = master_binding::bind(&catalog, entities) + .map_err(|error| error.safe_reason_code().to_string())?; + Ok(report.entities().iter().map(master_match_json).collect()) +} + +fn master_match_json(binding: &EntityBinding) -> Value { + let requested = party_name(binding.source_name.clone()); + match &binding.status { + BindingStatus::Bound { + catalog_name, + basis, + } => { + // Only byte-exact equality may be reported as `exact`: the import + // file carries the name verbatim, and build_import_xml admits + // nothing else. + let match_state = match basis { + BindingBasis::ExactName => "exact", + BindingBasis::NormalizedName => "normalized", + BindingBasis::Identifier => "identifier", + }; + json!({ + "requested": requested, + "match_state": match_state, + "exact_live_spelling": party_name(catalog_name.clone()), }) - .map(|name| party_name(name.clone())) - .collect::>(); - json!({"requested":party_name(wanted),"match_state":"near_miss", - "exact_live_spelling":candidates.first(),"candidate_count":candidate_count, - "candidates_truncated":candidates.len() < candidate_count,"candidates":candidates}) + } + BindingStatus::Ambiguous(unresolved) | BindingStatus::Unmatched(unresolved) => { + let mut bytes = 0_usize; + let candidates = unresolved + .candidates + .listed() + .iter() + .take_while(|candidate| { + bytes = bytes.saturating_add(candidate.catalog_name.len()); + bytes <= MAX_CANDIDATE_RESULT_BYTES + }) + .map(|candidate| { + json!({ + "name": party_name(candidate.catalog_name.clone()), + "rule": candidate.rule, + }) + }) + .collect::>(); + // The listing state is carried explicitly rather than left to be + // inferred from an empty array. A model is exactly the caller that + // would read "no candidates" as "no such ledger exists", and for + // `withheld` that is false: masters were found and deliberately not + // listed because none of them separates the requested name. + let found = unresolved.candidates.found(); + let listing = match unresolved.candidates { + Candidates::None => "none", + Candidates::Withheld { .. } => "withheld", + _ if candidates.len() < found => "truncated", + _ => "listed", + }; + // No `exact_live_spelling`. Naming one candidate as the live + // spelling is the auto-resolution that rejected a batch once. + json!({ + "requested": requested, + "match_state": match binding.status { + BindingStatus::Unmatched(_) => "missing", + _ => "near_miss", + }, + "reason": unresolved.reason.safe_reason_code(), + "listing": listing, + "candidate_count": found, + "candidates_truncated": listing != "listed" && listing != "none", + "candidates": candidates, + "unresolved_identity": unresolved + .unresolved_identity + .iter() + .map(|identifier| json!({ + "kind": identifier.kind, + "value": party_name(identifier.value.clone()), + })) + .collect::>(), + }) + } } } -fn master_key(value: &str) -> String { - value - .nfc() - .flat_map(|character| match character { - '–' | '—' | '−' | '‐' | '‑' => "-".chars().collect::>(), - '‘' | '’' | '‚' | '‛' => "'".chars().collect(), - '“' | '”' | '„' | '‟' => "\"".chars().collect(), - other => other.to_lowercase().collect(), - }) - .collect::() - .split_whitespace() - .collect::>() - .join(" ") -} - fn render_import_xml(company: &str, vouchers: &[ImportVoucher], batch_id: &str) -> String { let messages = vouchers .iter() diff --git a/src-tauri/src/agent_import_post.rs b/src-tauri/src/agent_import_post.rs index c8d093a58..58b0a0c1f 100644 --- a/src-tauri/src/agent_import_post.rs +++ b/src-tauri/src/agent_import_post.rs @@ -146,7 +146,7 @@ impl Server { .read_import_ledger_catalogue(&identity, &company.name) .await?; accumulated = combine_evidence(accumulated.clone(), evidence); - if masters_for_payload(&payload, &catalogue) + if masters_for_payload(&payload, &catalogue)? .iter() .any(|item| item["match_state"] != "exact") { diff --git a/src-tauri/src/agent_import_tests.rs b/src-tauri/src/agent_import_tests.rs index f52b60334..c3f45eaf6 100644 --- a/src-tauri/src/agent_import_tests.rs +++ b/src-tauri/src/agent_import_tests.rs @@ -278,29 +278,48 @@ fn schema_balance_matcher_rendering_and_ledger_append_are_fail_closed() { validate_payload(&unbalanced), Err("voucher_not_balanced".to_string()) ); + // ASCII case and one trailing space are transformations + // `TALLY_PROTOCOL_REFERENCE.md` §9.4b measured Tally performing, so they + // name the same live ledger: they bind and report its exact spelling. Only + // byte equality is `exact`, which is what build_import_xml admits. + for wanted in ["bank ", "bank", "BANK"] { + let matched = one_master_match(wanted, &["Bank"]); + assert_eq!(matched["match_state"], "normalized"); + assert_eq!( + matched["exact_live_spelling"][super::super::PARTY_NAME_MARKER], + "Bank" + ); + } + // A non-breaking space, an en dash and a curly quote are **not** on that + // list. Each still reaches its master through the wide fold, so the caller + // sees one candidate and confirms it; what it no longer gets is an answer + // and a live spelling to copy into the exact-only write gate. + for (wanted, live) in [ + ("A\u{a0}B", "A B"), + ("Fees-Admin", "Fees\u{2013}Admin"), + ("Bob's", "Bob\u{2019}s"), + ] { + let matched = one_master_match(wanted, &[live]); + assert_eq!( + matched["match_state"], "near_miss", + "{wanted} resolved on an unverified fold" + ); + assert!(matched.get("exact_live_spelling").is_none()); + assert_eq!( + matched["candidates"][0]["name"][super::super::PARTY_NAME_MARKER], + live + ); + } + assert_eq!(one_master_match("Bank", &["Bank"])["match_state"], "exact"); + // A shorter name that a live ledger extends is a near-miss with one + // candidate, and one candidate is still not a decision. + let near = one_master_match("Bank", &["Bank Charges"]); + assert_eq!(near["match_state"], "near_miss"); + assert_eq!(near["reason"], "master_binding_near_miss"); + assert!(near.get("exact_live_spelling").is_none()); assert_eq!( - master_match("bank ", &["Bank".to_string()])["match_state"], - "near_miss" - ); - assert_eq!( - master_match("bank", &["Bank".to_string()])["match_state"], - "near_miss" - ); - assert_eq!( - master_match("A\u{a0}B", &["A B".to_string()])["match_state"], - "near_miss" - ); - assert_eq!( - master_match("Fees-Admin", &["Fees–Admin".to_string()])["match_state"], - "near_miss" - ); - assert_eq!( - master_match("Bob's", &["Bob’s".to_string()])["match_state"], - "near_miss" - ); - assert_eq!( - master_match("Bank", &["Bank Charges".to_string()])["match_state"], - "near_miss" + near["candidates"][0]["name"][super::super::PARTY_NAME_MARKER], + "Bank Charges" ); let xml = render_import_xml("Book & Co", &input.vouchers, "batch-render"); assert!(xml.starts_with("")); @@ -1483,22 +1502,53 @@ fn native_captured_import_readback_keeps_direct_amounts_and_padded_identifiers() } } +/// Binds one name against a fabricated catalogue and returns its rendered row. +fn one_master_match(wanted: &str, catalogue: &[&str]) -> Value { + let catalogue = catalogue + .iter() + .map(|name| (*name).to_string()) + .collect::>(); + master_report( + &source_entities(&[wanted.to_string()]).expect("fabricated name parses"), + &catalogue, + ) + .expect("fabricated catalogue binds") + .remove(0) +} + #[test] fn master_match_bounds_suggestions_before_copying_names_and_preserves_ambiguity() { let catalogue = (0..100) .map(|index| format!("Ledger {index:03}")) .collect::>(); - let matched = master_match("L", &catalogue); + let borrowed = catalogue.iter().map(String::as_str).collect::>(); + // A name reaching a whole family distinguishes none of it. Measured against + // live books, listing an arbitrary capped slice omitted the right master + // about a third of the time, so the family is counted and not listed. + let matched = one_master_match("Ledger", &borrowed); assert_eq!(matched["match_state"], "near_miss"); + assert_eq!( + matched["reason"], + "master_binding_no_discriminating_candidate" + ); assert_eq!(matched["candidate_count"], 100); assert_eq!(matched["candidates_truncated"], true); - assert_eq!(matched["candidates"].as_array().unwrap().len(), 25); + assert!(matched["candidates"].as_array().unwrap().is_empty()); + // A family inside the bound is still listed in full. + let small = (0..5) + .map(|index| format!("Small {index:02}")) + .collect::>(); + let small_ref = small.iter().map(String::as_str).collect::>(); + let listed = one_master_match("Small", &small_ref); + assert_eq!(listed["candidates"].as_array().unwrap().len(), 5); assert_eq!( - master_match("Ledger 099", &catalogue)["match_state"], + one_master_match("Ledger 099", &borrowed)["match_state"], "exact" ); + // A single pathological live name is bounded by bytes before it is copied + // into a result, and its true count is still reported. let huge = format!("Large{}", "x".repeat(8192)); - let limited = master_match("L", std::slice::from_ref(&huge)); + let limited = one_master_match("Large", &[huge.as_str()]); assert_eq!(limited["match_state"], "near_miss"); assert_eq!(limited["candidate_count"], 1); assert_eq!(limited["candidates_truncated"], true); @@ -1506,6 +1556,57 @@ fn master_match_bounds_suggestions_before_copying_names_and_preserves_ambiguity( assert!(!limited.to_string().contains(&huge)); } +#[test] +fn a_catalogue_that_was_never_read_refuses_instead_of_reporting_everything_missing() { + // "Nobody read the ledger list out of Tally first" is the recorded cause + // of the one failed engagement, so an empty catalogue must not look like + // an answer. P5: nothing-found and request-failed stay distinguishable. + assert_eq!( + master_report(&source_entities(&["Bank".to_string()]).expect("valid"), &[]), + Err("master_catalog_empty".to_string()) + ); +} + +#[test] +fn an_embedded_identifier_decides_where_the_name_offers_wrong_candidates() { + // Fabricated from a placeholder alphabet: the live ledger carries a number + // the operator typed into its name, and the requested name matches no live + // spelling. The number is the key; the name is a hint. + let matched = one_master_match( + "GAMMA. EPSILON 5550000001", + &["GAMMA (5550000001)", "GAMMA ALPHA", "GAMMA BETA"], + ); + assert_eq!(matched["match_state"], "identifier"); + assert_eq!( + matched["exact_live_spelling"][super::super::PARTY_NAME_MARKER], + "GAMMA (5550000001)" + ); +} + +#[test] +fn a_near_miss_never_names_a_live_spelling_and_retains_its_identity() { + let matched = one_master_match( + "PARTY 5550000001", + &["ALPHA (5550000001)", "BETA (5550000001)"], + ); + assert_eq!(matched["match_state"], "near_miss"); + assert_eq!(matched["reason"], "master_binding_identifier_conflict"); + assert!(matched.get("exact_live_spelling").is_none()); + assert_eq!(matched["candidate_count"], 2); + assert_eq!( + matched["unresolved_identity"][0]["value"][super::super::PARTY_NAME_MARKER], + "5550000001" + ); +} + +#[test] +fn nothing_defensible_is_reported_missing_with_no_candidate() { + let matched = one_master_match("Zeta Placeholder", &["Bank", "Cash"]); + assert_eq!(matched["match_state"], "missing"); + assert_eq!(matched["reason"], "master_binding_no_candidate"); + assert!(matched["candidates"].as_array().unwrap().is_empty()); +} + #[tokio::test] async fn import_bounds_distinct_ledger_names_before_tally_without_reducing_voucher_limit() { let mut repeated = payload(); diff --git a/src-tauri/src/source_draft/catalog.rs b/src-tauri/src/source_draft/catalog.rs index f58022c84..2a7b7f214 100644 --- a/src-tauri/src/source_draft/catalog.rs +++ b/src-tauri/src/source_draft/catalog.rs @@ -9,6 +9,9 @@ use std::collections::BTreeMap; use serde::{Deserialize, Serialize}; use uuid::Uuid; +use bridge_tally_core::master_binding::{ + self, BindingBasis, BindingStatus, MasterCatalog, MasterClass, SourceEntity, +}; use bridge_tally_protocol::{StandardLedgerCatalog, StandardLedgerCatalogBinding}; use crate::{ @@ -52,9 +55,36 @@ pub(crate) struct SourceDraftCatalogTargets { pub(crate) capture_id: String, pub(crate) source_sha256: String, pub(crate) targets: Vec, + pub(crate) bindings: Vec, + /// `complete` when every source entry was bound, `unavailable` when the + /// narrowing pass could not run. An empty `bindings` list is otherwise + /// indistinguishable from a failed one, and the catalogue read itself still + /// succeeded. + pub(crate) bindings_state: &'static str, pub(crate) evidence: SourceDraftCatalogEvidence, } +/// One source entry's deterministic binding against the capture, so an +/// operator sees the few relevant ledgers rather than the whole catalog. +/// +/// This narrows a list and grants nothing. `bound_target` names a live ledger +/// only where the rules in `bridge_tally_core::master_binding` decided it +/// outright; a near-miss carries candidates and no target. Applying any of +/// them still goes through the unchanged apply path, which rereads the catalog +/// and proves the selection is current — matching text remains never a +/// selected or approved target. +#[derive(Debug, Serialize)] +pub(crate) struct SourceDraftCatalogBinding { + pub(crate) row_position: usize, + pub(crate) entry_position: usize, + pub(crate) bound_target: Option, + pub(crate) bound_basis: Option, + pub(crate) unbound_reason: Option<&'static str>, + pub(crate) candidates: Vec, + pub(crate) candidate_count: usize, + pub(crate) candidates_truncated: bool, +} + #[derive(Debug, Serialize)] pub(crate) struct SourceDraftCatalogEvidence { pub(crate) request_sha256: String, @@ -107,6 +137,85 @@ pub(super) struct CatalogApplySnapshot { pub(super) catalog: StandardLedgerCatalog, } +/// Binds every source entry's observed ledger name against the captured +/// catalog. Advisory only: an empty or unusable capture narrows nothing rather +/// than failing the read the operator just performed, and every returned name +/// is still revalidated by the apply path before it can become a target. +fn source_entry_bindings( + source: &crate::source_draft_xml::ParsedSource, + targets: &[String], +) -> (Vec, &'static str) { + let Ok(catalog) = MasterCatalog::new(MasterClass::Ledger, targets) else { + return (Vec::new(), "unavailable"); + }; + let mut located = Vec::new(); + let mut entities = Vec::new(); + for voucher in &source.vouchers { + for entry in &voucher.entries { + // Dropping an unusable entry would return fewer bindings than the + // source has rows while still claiming completeness, and the row + // that vanished is exactly the one an operator needs to look at. + let Ok(entity) = SourceEntity::new(entities.len(), &entry.ledger) else { + return (Vec::new(), "unavailable"); + }; + located.push((voucher.position, entry.position)); + entities.push(entity); + } + } + let Ok(report) = master_binding::bind(&catalog, &entities) else { + // A refusal is reported as such. Returning an empty list here would let + // a failed pass read exactly like a source that narrowed to nothing. + return (Vec::new(), "unavailable"); + }; + // The report itself is bounded by `MAX_REPORT_CANDIDATE_BYTES`, so this + // path no longer needs a second budget of its own: capping the copy left + // the original allocation unbounded, which was the actual stall risk. + let bindings = report + .entities() + .iter() + .zip(located) + .map( + |(binding, (row_position, entry_position))| match &binding.status { + BindingStatus::Bound { + catalog_name, + basis, + } => SourceDraftCatalogBinding { + row_position, + entry_position, + bound_target: Some(catalog_name.clone()), + bound_basis: Some(*basis), + unbound_reason: None, + candidates: Vec::new(), + candidate_count: 0, + candidates_truncated: false, + }, + BindingStatus::Ambiguous(unresolved) | BindingStatus::Unmatched(unresolved) => { + SourceDraftCatalogBinding { + row_position, + entry_position, + bound_target: None, + bound_basis: None, + unbound_reason: Some(unresolved.reason.safe_reason_code()), + // The screen distinguishes the three cases from + // `candidate_count` against an empty list and is tested + // on each, so the DTO stays flat and this projection is + // the only place the typed shape is flattened. + candidates_truncated: unresolved.candidates.is_incomplete(), + candidates: unresolved + .candidates + .listed() + .iter() + .map(|candidate| candidate.catalog_name.clone()) + .collect(), + candidate_count: unresolved.candidates.found(), + } + } + }, + ) + .collect(); + (bindings, "complete") +} + /// The freshly read catalog must still contain the selected pair. The response /// is parsed once by the read itself, so this takes the parsed catalog rather /// than reparsing the body per binding. @@ -253,6 +362,7 @@ impl SourceDraftStore { return Err(error("source_draft_catalogue_invalidated")); } let targets = read.catalog.names().map(str::to_owned).collect::>(); + let (bindings, bindings_state) = source_entry_bindings(¤t.source, &targets); let capture = CatalogCapture { id: Uuid::new_v4(), draft_id: current.id, @@ -267,6 +377,8 @@ impl SourceDraftStore { capture_id: capture.id.to_string(), source_sha256: capture.source_sha256.clone(), targets, + bindings, + bindings_state, evidence: SourceDraftCatalogEvidence { request_sha256: read.request_sha256, response_sha256: read.response_sha256, @@ -479,6 +591,69 @@ mod tests { Fixture, ProductStatus, ResponseFraming, ScenarioPlan, SequenceSimulator, WireEncoding, }; + /// Fabricated from a placeholder alphabet; nothing here is edited down + /// from an observed book. + fn fabricated_source() -> crate::source_draft_xml::ParsedSource { + parse_source_xml( + concat!( + "", + "20260901", + "alpha traders1", + "GAMMA. EPSILON 5550000001-1", + "Zeta Placeholder0", + "" + ) + .as_bytes(), + "source.xml".into(), + ) + .expect("fabricated source parses") + } + + #[test] + fn a_capture_narrows_each_source_entry_without_deciding_a_near_miss() { + let targets = [ + "Alpha Traders".to_string(), + "GAMMA (5550000001)".to_string(), + "GAMMA ALPHA".to_string(), + "Beta Supply".to_string(), + ]; + let (bindings, state) = source_entry_bindings(&fabricated_source(), &targets); + assert_eq!(state, "complete"); + assert_eq!(bindings.len(), 3); + + // Case alone does not defeat a bind, and the live spelling is named. + assert_eq!(bindings[0].row_position, 1); + assert_eq!(bindings[0].entry_position, 1); + assert_eq!(bindings[0].bound_target.as_deref(), Some("Alpha Traders")); + assert_eq!(bindings[0].bound_basis, Some(BindingBasis::NormalizedName)); + + // The number the operator buried in the ledger name decides where the + // name offers a wrong candidate. + assert_eq!( + bindings[1].bound_target.as_deref(), + Some("GAMMA (5550000001)") + ); + assert_eq!(bindings[1].bound_basis, Some(BindingBasis::Identifier)); + + // Nothing defensible stays unbound with no target of any kind. + assert!(bindings[2].bound_target.is_none()); + assert_eq!( + bindings[2].unbound_reason, + Some("master_binding_no_candidate") + ); + assert!(bindings[2].candidates.is_empty()); + } + + #[test] + fn a_narrowing_pass_that_could_not_run_says_so_rather_than_looking_empty() { + // An unusable capture narrows nothing rather than discarding a read the + // operator just performed — but "no bindings" and "binding failed" must + // not read alike, because the catalogue read itself still succeeded. + let (bindings, state) = source_entry_bindings(&fabricated_source(), &[]); + assert!(bindings.is_empty()); + assert_eq!(state, "unavailable"); + } + const CAPTURED_COMPANY: &str = "WR2 Unicode Lab"; const CAPTURED_GUID: &str = "61c6de69-1748-461c-ad3f-162cb949df9f"; diff --git a/src/source-draft-types.ts b/src/source-draft-types.ts index df7aee387..73e458c82 100644 --- a/src/source-draft-types.ts +++ b/src/source-draft-types.ts @@ -65,10 +65,36 @@ export type SourceDraftCompanyScope = { }; }; +/// One source entry's deterministic binding against the captured catalog. +/// Advisory: it narrows the target list and confers no authority. Applying a +/// name still goes through the unchanged assign path, which rereads the +/// catalog and proves the selection is current. +export type SourceDraftCatalogBinding = { + row_position: number; + entry_position: number; + bound_target: string | null; + bound_basis: "identifier" | "exact_name" | "normalized_name" | null; + unbound_reason: string | null; + candidates: string[]; + candidate_count: number; + candidates_truncated: boolean; +}; + export type SourceDraftCatalogTargets = { capture_id: string; source_sha256: string; targets: string[]; + bindings: SourceDraftCatalogBinding[]; + /// Whether the narrowing pass **ran**, not whether it resolved anything. + /// "complete" means every source entry was put through binding and carries a + /// result — which for many of them will be a near miss or nothing at all; + /// "unavailable" means the pass could not run, so an empty list says nothing. + /// An empty list alone cannot distinguish the two, which is why this exists. + /// + /// It is **not** an all-bound signal and must not gate resolution: a draft + /// whose every entry is unmatched still reports "complete". Read the bindings + /// for that. + bindings_state: "complete" | "unavailable"; evidence: { request_sha256: string; response_sha256: string; bytes: number; state: "complete" }; }; diff --git a/tools/Cargo.lock b/tools/Cargo.lock index 70443a14f..e9765496c 100644 --- a/tools/Cargo.lock +++ b/tools/Cargo.lock @@ -92,6 +92,7 @@ dependencies = [ "serde_json", "sha2", "thiserror", + "unicode-normalization", ] [[package]] @@ -1648,6 +1649,15 @@ version = "1.0.24" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" +[[package]] +name = "unicode-normalization" +version = "0.1.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5fd4f6878c9cb28d874b009da9e8d183b5abc80117c40bbd187a1fde336be6e8" +dependencies = [ + "tinyvec", +] + [[package]] name = "untrusted" version = "0.9.0" diff --git a/tools/bridge-tally-compatibility/src/lib.rs b/tools/bridge-tally-compatibility/src/lib.rs index dee2ff771..a315fe61f 100644 --- a/tools/bridge-tally-compatibility/src/lib.rs +++ b/tools/bridge-tally-compatibility/src/lib.rs @@ -30,7 +30,20 @@ pub const RESERVED_SURFACE_FILES: usize = 15; /// reserved capacity covers a small cohesive feature (source, tests, docs /// and manifest) but makes further unreviewed additions an explicit /// compatibility-surface decision. -pub const MAX_SURFACE_FILES: usize = 211; +/// +/// **Raised twice, by two branches that did not see each other.** 210 to 211 on +/// master for `src-tauri/src/agent_ledgers.rs`, and 211 to 212 here for +/// `src-tauri/crates/bridge-tally-core/src/master_binding.rs`. Both reasons +/// stand and the number carries both; a merge that kept one raise and one pin +/// would pass the gate with the other file silently unpinned, which is the +/// failure this constant exists to make loud. +/// +/// `master_binding.rs` decides `validate_masters` results and, through them, +/// import admission. Left unpinned, an edit confined to the matcher would leave +/// the surface digest unchanged and let existing evidence attest behaviour it +/// never covered. That is the deliberate decision the paragraph above requires, +/// and it is one file for one named reason — not headroom. +pub const MAX_SURFACE_FILES: usize = 212; pub const MAX_OPERATIONS: usize = 16; pub const MAX_CLAIMS: usize = 128; pub const MAX_KEYS: usize = 32;