diff --git a/SPEC.md b/SPEC.md index 8457d46..ad3192d 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,8 +1,8 @@ # Technical Specification: OctetProof — A License-Free Verification Protocol for Undocumented Binary Formats -**Version:** 1.1.0 +**Version:** 1.1.1 **Date:** 2026-08-30 -**Status:** Specification — 1.1.0. Supersedes the draft received from the project owner on 2026-08-30, which is retained verbatim at [`docs/octetproof-spec-draft.md`](octetproof-spec-draft.md) with its reviewer notes. The corrections applied to reach 1.0.0 are listed in Section 19; the additive 1.1.0 change is Section 20. +**Status:** Specification — 1.1.1. Supersedes the draft received from the project owner on 2026-08-30, which is retained verbatim at [`docs/octetproof-spec-draft.md`](octetproof-spec-draft.md) with its reviewer notes. The corrections applied to reach 1.0.0 are listed in Section 19; the additive 1.1.0 change is Section 20; the non-semantic 1.1.1 change is Section 20a. **License of this document:** CC-BY-4.0 **Reference implementation:** rvt-rs (Apache-2.0) — in-repo instance; umbrella repository: [DrunkOnJava/octetproof](https://github.com/DrunkOnJava/octetproof) **Primary domain:** Building Information Modeling (BIM) closed formats, with generalization to any undocumented binary container format @@ -215,7 +215,7 @@ The reference implementation carries the registry as `research/witness-registry. |— |`checked` |Extension: the date this registry verified the entry against the upstream API. Absent means unverified. | |— |`notes` |Extension; the registry's `candidate_claims` rule applies — any figure quoted here is the project's own claim | |`ci_eligible` |not implemented |**Umbrella scope.** The in-repo gate runs a fixed witness set named by the CI job rather than filtering the registry. | -|`coverage` / §9.4 coverage declaration |not implemented |**Umbrella scope.** The per-run claim lives in the observation's `semantic_surface_covered`; there is no registry-level declaration to check it against.| +|`coverage` / §9.4 coverage declaration |`covers` |Implemented in the registry — an array drawn from the §9.4 vocabulary, declared by every adopted reader and by readers only, since coverage is a claim about reading and an authoring witness emits an artifact rather than an observation. `tests/witness_registry.rs` requires every committed observation's `semantic_surface_covered` to be a subset of its witness's `covers`, so the per-run claim can never exceed the registered one.| |`versions` |not implemented |**Umbrella scope.** `node` names the format, not the supported version range. | |§9.5 determinism attestation |not implemented |**Umbrella scope.** The observation carries a per-run `deterministic` flag; there is no cross-OS, cross-architecture attestation. | |§9.6 exact version pinning |not implemented |**Umbrella scope.** The observation carries `witness_version`; CI pins IfcOpenShell to a version range, not a commit. | @@ -257,7 +257,7 @@ Every example in this section is valid JSON and matches the reference implementa - [`docs/schemas/witness-observation.schema.json`](schemas/witness-observation.schema.json) - [`docs/schemas/witness-verdict.schema.json`](schemas/witness-verdict.schema.json) -The worked example throughout is the first committed artifact, `magnetar-2024-core-interior`: the RVT → IFC edge, with `rvt-rs` as the source witness and IfcOpenShell as the bridge witness. The files quoted below are the real committed ones under `research/witness/magnetar-2024-core-interior/`. +The worked example throughout is the RVT → IFC edge of the magnetar Core Interior project, with `rvt-rs` as the source witness reading the `.rvt` and IfcOpenShell and IFClite as two independent bridge witnesses reading Revit's own `.ifc`. Section 6.1 quotes the manifest of the first recorded artifact, `magnetar-2024-core-interior` (the 20 KB element-export fixture). Sections 6.2 and 6.3 quote the full-project artifact `magnetar-2024-core-interior-slim`, whose observations and verdict are the ones the gate emits today: three witnesses, three implementation lineages, and a thirteen-field surface spanning all three field classes. All of them are the real committed files under `research/witness/`, as of commit `5da2570` (2026-08-30) — **dated snapshots of a decoder state**, not a live mirror (see 19b and 20a). ### 6.1 Golden Artifact Manifest @@ -366,7 +366,7 @@ An observation is what one witness saw in one input, plus enough provenance to r |Key |Type |Meaning | |---------------------------|----------------|------------------------------------------------------------------------------------------------------------| -|`schema_version` |string |`"1.0.0"` for this specification | +|`schema_version` |string |`"1.0.0"`, or `"1.1.0"` for an observation carrying a field class added in Section 20 | |`witness_id` |string |Registry id; must resolve in the registry when the gate runs with one | |`witness_version` |string |The exact version that produced this observation | |`artifact_id` |string |The manifest `id` | @@ -380,36 +380,72 @@ An observation is what one witness saw in one input, plus enough provenance to r |`unsupported_entities` |array of string |What the witness could not read; recorded, never a disagreement (Section 7.1 rule 4) | |`warnings` |array of string |Non-fatal notes from the run | -The bridge witness, IfcOpenShell reading the Revit-authored IFC — the complete committed file: +The bridge witness, IfcOpenShell reading the Revit-authored IFC — the committed file `research/witness/magnetar-2024-core-interior-slim/observations/ifcopenshell.json` at commit `5da2570` (2026-08-30), verbatim except that the two pair arrays are trimmed for length: `relations.IFCRELFILLSELEMENT` shows 3 of its 138 pairs and `storeys.IFCBUILDINGSTOREY` 3 of its 15. Nothing else is abbreviated, and no key is omitted: ```json { - "artifact_id": "magnetar-2024-core-interior", + "artifact_id": "magnetar-2024-core-interior-slim", "deterministic": true, - "input_file": "2024_Core_Interior.ifc", - "input_hash_sha256": "d07c7462aee22640661faed5262cf802ce0fcbc663f312961a39be92bf857050", + "input_file": "../IFC Exports/2024_Core_Interior_slim.ifc", + "input_hash_sha256": "bfdf36ffb0bb768f3409d818403990e64d4c262c6780603be87f8077387ad86d", "input_role": "bridge", "observation": { "entity_counts": { "IFCBEAM": 0, - "IFCCOLUMN": 0, - "IFCDOOR": 0, + "IFCBUILDINGSTOREY": 15, + "IFCCOLUMN": 256, + "IFCDOOR": 132, "IFCFLOWTERMINAL": 0, - "IFCMATERIAL": 1, - "IFCPROPERTYSET": 25, + "IFCMATERIAL": 10, + "IFCPROPERTYSET": 0, "IFCROOF": 0, - "IFCSHADINGDEVICE": 1, - "IFCSPACE": 0, + "IFCSHADINGDEVICE": 20, + "IFCSLAB": 80, + "IFCSPACE": 116, "IFCUNITASSIGNMENT": 1, - "IFCWALL": 0, - "IFCWINDOW": 0 + "IFCWALL": 360, + "IFCWINDOW": 6 }, - "ifc_schema": "IFC4" + "ifc_schema": "IFC4", + "relations": { + "IFCRELFILLSELEMENT": [ + [ + "20796", + "20827" + ], + [ + "20798", + "20810" + ], + [ + "20798", + "20815" + ] + ] + }, + "storeys": { + "IFCBUILDINGSTOREY": [ + [ + "Basement 1", + "-20.000000" + ], + [ + "Basement 2", + "-40.000000" + ], + [ + "Level 1", + "0.000000" + ] + ] + } }, - "observation_hash_sha256": "8cf4046509bb788406e93261f0dcb10708fbf33ae195d12f8ddc99b41a93398f", - "schema_version": "1.0.0", + "observation_hash_sha256": "5a101408488d591d4e7d15fe39969f1ee295c282dfe53c1f093468ae3e5cb2d0", + "schema_version": "1.1.0", "semantic_surface_covered": [ - "entity_counts" + "entity_counts", + "relations", + "storeys" ], "unsupported_entities": [], "warnings": [], @@ -418,52 +454,99 @@ The bridge witness, IfcOpenShell reading the Revit-authored IFC — the complete } ``` -The source witness, rvt-rs reading the `.rvt` directly — the same file, abbreviated in the `entity_counts`, `unsupported_entities`, and `warnings` arrays only: +The third witness, IFClite reading the same bytes with its own STEP scanner, is not reproduced here: its payload canonicalizes to the same hash `5a101408…`, so the file differs only in `witness_id` and `witness_version`. Two unrelated implementations reporting a bit-identical payload is the agreement the protocol is for. + +The source witness, rvt-rs reading the `.rvt` directly — the committed `observations/rvt-rs.json` from the same directory and commit, abbreviated in four places: `entity_counts` shows 12 of its 43 types, the two pair arrays show the same 3 of 138 and 3 of 15 as above, `unsupported_entities` 2 of 5, and `warnings` 1 of 5. Every scalar and every other key is verbatim. The 31 types dropped from `entity_counts` are the STEP scaffolding rvt-rs emits and the manifest does not name — `IFCCARTESIANPOINT`, `IFCLOCALPLACEMENT`, and the like; `IFCROOF`, `IFCBEAM` and `IFCFLOWTERMINAL` are absent from the full file too, and that absence is load-bearing below: ```json { - "artifact_id": "magnetar-2024-core-interior", + "artifact_id": "magnetar-2024-core-interior-slim", "deterministic": true, "input_file": "2024_Core_Interior.rvt", "input_hash_sha256": "c805df445d613b408e37337765572021265e3f5dfdc7d1fa53b22ba1600b8014", "input_role": "source", "observation": { - "building_elements_with_geometry": 0, + "building_elements_with_geometry": 854, "entity_counts": { - "IFCBUILDINGSTOREY": 12, + "IFCBUILDINGSTOREY": 15, + "IFCCOLUMN": 256, + "IFCDOOR": 132, "IFCMATERIAL": 102, - "IFCPROPERTYSET": 64, - "IFCSLAB": 64, + "IFCPROPERTYSET": 854, + "IFCRELFILLSELEMENT": 138, + "IFCSHADINGDEVICE": 20, + "IFCSLAB": 80, "IFCSPACE": 18, - "IFCUNITASSIGNMENT": 1 + "IFCUNITASSIGNMENT": 1, + "IFCWALL": 360, + "IFCWINDOW": 6 }, "exported_building_elements": { - "IFCSLAB": 64, - "IFCSPACE": 18 + "IFCCOLUMN": 256, + "IFCDOOR": 132, + "IFCSHADINGDEVICE": 20, + "IFCSLAB": 80, + "IFCSPACE": 18, + "IFCWALL": 360, + "IFCWINDOW": 6 }, "material_count": 102, - "storey_count": 12 + "relations": { + "IFCRELFILLSELEMENT": [ + [ + "20796", + "20827" + ], + [ + "20798", + "20810" + ], + [ + "20798", + "20815" + ] + ] + }, + "storey_count": 15, + "storeys": { + "IFCBUILDINGSTOREY": [ + [ + "Basement 1", + "-20.000000" + ], + [ + "Basement 2", + "-40.000000" + ], + [ + "Level 1", + "0.000000" + ] + ] + } }, - "observation_hash_sha256": "b6d9b67c10c3350b69b58cbd2c6caeca405f63ce182e7291feba3e3ff10f3e00", - "schema_version": "1.0.0", + "observation_hash_sha256": "70f8df9dd188b7e42947bfa167b2810333b3d0dc4398019ea40570b7fcb87c24", + "schema_version": "1.1.0", "semantic_surface_covered": [ - "entity_counts" + "entity_counts", + "relations", + "storeys" ], "unsupported_entities": [ - "real_file_element_geometry", - "floor_slab_extrusion_thickness" + "partial_element_geometry", + "revit_compound_assemblies_and_walltype_widths" ], "warnings": [ - "No exported building elements include decoded geometry; see skipped diagnostics for missing curve/profile/dimension data." + "Unsupported or incomplete geometry was reported: unsupported_geometry_curve=18, unsupported_geometry_missing_level=71, unsupported_geometry_missing_dimensions=18." ], "witness_id": "rvt-rs", "witness_version": "0.1.2" } ``` -The committed hash `b6d9b6…` is over the full payload, not the abbreviation above; the abbreviated block is illustrative of shape only. Every other example in this section was byte-exact against the files committed on 2026-08-30; they are **dated snapshots of a decoder state**, not a live mirror of `research/witness/` (see 19b). +Both committed hashes are over the full payload, not the abbreviations above; where an array is trimmed the block is illustrative of shape only, and `5a101408…` and `70f8df9d…` are the hashes of the untrimmed files. The examples in this section are **dated snapshots of a decoder state**, not a live mirror of `research/witness/` (see 19b and 20a). -Since 1.1.0 the payload may also carry `relations`, an object mapping a relation type to its sorted pair multiset: +Both observations declare `schema_version` `"1.1.0"`, because both carry the two field classes Section 20 adds. `relations` is an object mapping a relation type to its sorted pair multiset: ```json "relations": { @@ -471,9 +554,9 @@ Since 1.1.0 the payload may also carry `relations`, an object mapping a relation } ``` -A witness emitting it declares `relations` in `semantic_surface_covered` and sets `schema_version` to `"1.1.0"`; the examples below predate that and are 1.0.0 snapshots. +A witness emitting it declares `relations` in `semantic_surface_covered` and sets `schema_version` to `"1.1.0"`. A 1.0.0 observation carries neither key and remains valid input to a 1.1.0 gate. -Since 1.1.0 the payload may also carry `storeys`, an object mapping a spatial type to its sorted `[name, elevation-in-feet]` set: +`storeys` is an object mapping a spatial type to its sorted `[name, elevation-in-feet]` set: ```json "storeys": { @@ -481,9 +564,9 @@ Since 1.1.0 the payload may also carry `storeys`, an object mapping a spatial ty } ``` -A witness emitting it declares `storeys` in `semantic_surface_covered`. The elevation is a string in feet, unit-normalized by each witness from its own file's declared `LENGTHUNIT` (Section 7.2, field class *storey sets*). +A witness emitting it declares `storeys` in `semantic_surface_covered`. The elevation is a string in feet, unit-normalized by each witness from its own file's declared `LENGTHUNIT` (Section 7.2, field class *storey sets*) — here from Revit's `FOOT` on the bridge side and from rvt-rs's own `METRE` on the source side, which is why both print `-20.000000` for `Basement 1`. -Two properties of the payload are load-bearing. First, the witnesses share only what the manifest names — `entity_counts`, and since 1.1.0 `relations` and `storeys`; everything else in a payload is witness-specific and is not diffed. Second, a witness may report a type the other never emits (`IFCSHADINGDEVICE` here) — the diff is driven by the manifest's declared surface, not by the union of the payload keys, and a type absent from a payload counts as zero. +Three properties of the payload are load-bearing. First, the witnesses share only what the manifest names — `entity_counts`, and since 1.1.0 `relations` and `storeys`; everything else in a payload is witness-specific and is not diffed, which is why `building_elements_with_geometry`, `exported_building_elements`, `material_count` and `storey_count` appear on one side and nowhere in the verdict. Second, a witness may report a type the other never emits (`IFCROOF`, `IFCBEAM` and `IFCFLOWTERMINAL` here, which the bridge reports as zero and the source payload omits entirely) — the diff is driven by the manifest's declared surface, not by the union of the payload keys, and a type absent from a payload counts as zero. Third, what a witness may claim is bounded by its registry `covers` (Section 9.4): all three witnesses here declare `entity_counts`, `relations` and `storeys`, and an observation claiming more than its registration would be rejected before the diff runs. Observations are **canonicalized** with RFC 8785 (JSON Canonicalization Scheme) before hashing or diffing. Key ordering, number formatting, and Unicode normalization are fixed. This guarantees bit-identical hashes across languages and platforms. @@ -498,7 +581,7 @@ The verdict is the gate's output for one artifact. It records the whole decision |`status` |yes |One of the Section 10.5 vocabulary | |`witnesses_compared` |yes |Sorted witness ids remaining in the gate after commercial witnesses are dropped | |`inputs` |yes |Per witness: `input_hash_sha256`, resolved `role`, `witness_version` | -|`semantic_surface` |yes |The fields actually compared, as `entity_counts.` | +|`semantic_surface` |yes |The fields actually compared, as `.` — `entity_counts.`, and since 1.1.0 `relations.` and `storeys.`| |`excluded` |yes |Fields removed from the surface first-class, with reason and tracking issue | |`diffs` |yes |Pairwise disagreements inside the surface | |`insufficient_witnesses`|yes |True when fewer than two observations were present | @@ -509,26 +592,19 @@ The verdict is the gate's output for one artifact. It records the whole decision |`verdict_hash_sha256` |yes |SHA-256 over the canonicalized verdict excluding `timestamp` | |`timestamp` |optional|ISO-8601. Omitted by default so the verdict is byte-reproducible. | -The passing verdict for the worked example as committed on 2026-08-30, complete (a dated snapshot — see 19b): +The passing verdict for the worked example — `research/witness/magnetar-2024-core-interior-slim/verdict.json` at commit `5da2570` (2026-08-30), complete, nothing trimmed (a dated snapshot — see 19b and 20a): ```json { - "artifact_id": "magnetar-2024-core-interior", + "artifact_id": "magnetar-2024-core-interior-slim", "diffs": [], "excluded": [ - { - "category": "floors", - "field": "entity_counts.IFCSHADINGDEVICE", - "reason": "known_gap", - "tracking_issue": 31, - "unsupported_feature": "floor_slab_extrusion_thickness" - }, { "category": "rooms_spaces", "field": "entity_counts.IFCSPACE", "reason": "known_gap", "tracking_issue": 33, - "unsupported_feature": "typed_door_window_discrimination_and_host_binding" + "unsupported_feature": "partial_element_geometry" }, { "category": "materials", @@ -542,12 +618,13 @@ The passing verdict for the worked example as committed on 2026-08-30, complete "field": "entity_counts.IFCPROPERTYSET", "reason": "known_gap", "tracking_issue": 35, - "unsupported_feature": "typed_door_window_discrimination_and_host_binding" + "unsupported_feature": "revit_element_parameters_to_ifc_property_sets" } ], "independence": { "commercial_dropped": [], "lineages": [ + "ifc-lite", "ifcopenshell", "rvt-rs" ], @@ -559,8 +636,13 @@ The passing verdict for the worked example as committed on 2026-08-30, complete "strong_copyleft": [] }, "inputs": { + "ifc-lite": { + "input_hash_sha256": "bfdf36ffb0bb768f3409d818403990e64d4c262c6780603be87f8077387ad86d", + "role": "bridge", + "witness_version": "7.1.1" + }, "ifcopenshell": { - "input_hash_sha256": "d07c7462aee22640661faed5262cf802ce0fcbc663f312961a39be92bf857050", + "input_hash_sha256": "bfdf36ffb0bb768f3409d818403990e64d4c262c6780603be87f8077387ad86d", "role": "bridge", "witness_version": "0.8.5" }, @@ -573,37 +655,43 @@ The passing verdict for the worked example as committed on 2026-08-30, complete "insufficient_witnesses": false, "schema_version": "1.0.0", "semantic_surface": [ + "entity_counts.IFCBUILDINGSTOREY", "entity_counts.IFCWALL", + "entity_counts.IFCSLAB", "entity_counts.IFCROOF", "entity_counts.IFCDOOR", "entity_counts.IFCWINDOW", "entity_counts.IFCCOLUMN", "entity_counts.IFCBEAM", "entity_counts.IFCFLOWTERMINAL", - "entity_counts.IFCUNITASSIGNMENT" + "entity_counts.IFCUNITASSIGNMENT", + "entity_counts.IFCSHADINGDEVICE", + "relations.IFCRELFILLSELEMENT", + "storeys.IFCBUILDINGSTOREY" ], "status": "PASS", - "verdict_hash_sha256": "3df3ba73bb3f86fd3fab63c4a4db5fb7074fa18036056f01bba45d0c0b43f53a", + "verdict_hash_sha256": "cd171b3167c602d0882dd8a0d38c99791012a9fe693043e43e2d43b028554325", "witnesses_compared": [ + "ifc-lite", "ifcopenshell", "rvt-rs" ] } ``` -This is what an honest thin verdict looks like: eight fields inside the surface, of which seven are zero counts, and four categories excluded as tracked decoder gaps. The verdict says so rather than rounding up to "verified". +Read the whole record, not the `PASS`. Thirteen fields are inside the surface and three categories are excluded first-class as tracked decoder gaps, each named with its issue. The surface spans all three field classes: eleven counts (of which `IFCROOF`, `IFCBEAM` and `IFCFLOWTERMINAL` are zero on both sides), one relation of 138 pairs, and one storey set of 15 `[name, elevation]` pairs. `independence.lineages` carries three unrelated implementations — a Rust source reader and two bridge readers in C++/Python and Rust — with `strong_copyleft` and `commercial_dropped` both empty, so Section 9.3 is satisfied with margin rather than by exactly meeting the floor. What the verdict does *not* claim is equally explicit: `IFCSPACE`, `IFCMATERIAL` and `IFCPROPERTYSET` are measured gaps between rvt-rs and Revit's own exporter, recorded rather than rounded up to "verified". On disagreement, `status` becomes `DISAGREE` and `diffs` carries one entry per disagreeing pair per field. `tolerance_applied` records the tolerance the comparison actually used, taken from the manifest category (`0` for an exact field): ```json { - "artifact_id": "magnetar-2024-core-interior", + "artifact_id": "magnetar-2024-core-interior-slim", "diffs": [ { "field": "entity_counts.IFCWALL", "tolerance_applied": 0, - "value_a": 0, - "value_b": 3, + "value_a": 360, + "value_b": 357, "witness_a": "ifcopenshell", "witness_b": "rvt-rs" } @@ -617,6 +705,7 @@ On disagreement, `status` becomes `DISAGREE` and `diffs` carries one entry per d "status": "DISAGREE", "verdict_hash_sha256": "0000000000000000000000000000000000000000000000000000000000000000", "witnesses_compared": [ + "ifc-lite", "ifcopenshell", "rvt-rs" ], @@ -624,13 +713,16 @@ On disagreement, `status` becomes `DISAGREE` and `diffs` carries one entry per d } ``` +This example is constructed, not committed: the two hypothetical values stand in for a regression, and the committed verdict above carries `diffs: []`. Note that a third agreeing witness does not dissolve a disagreement — `diffs` is pairwise, so one dissenting reader is enough to fail the gate. + On replay, the gate re-hashes each fresh payload against the committed observation of the same witness and records the outcome per witness. Any value other than `match` sets `REPLAY_DRIFT`. The two keys that change are shown below; the rest of the verdict is as above: ```json { "replay": { + "ifc-lite": "match", "ifcopenshell": "match", - "rvt-rs": "drift (committed b6d9b67c10c3350b69b58cbd2c6caeca405f63ce182e7291feba3e3ff10f3e00, fresh 0000000000000000000000000000000000000000000000000000000000000000)" + "rvt-rs": "drift (committed 70f8df9dd188b7e42947bfa167b2810333b3d0dc4398019ea40570b7fcb87c24, fresh 0000000000000000000000000000000000000000000000000000000000000000)" }, "status": "REPLAY_DRIFT" } @@ -803,7 +895,9 @@ Every registered witness must declare, in the registry, the **semantic surface** The same vocabulary is what an observation's `semantic_surface_covered` array and a manifest's `faithful_surface` / `explicitly_excluded` arrays draw from. Witness identifiers are never members of it. -A witness may only be compared on fields it declares. Declaring a field it cannot actually parse is a registration violation and grounds for removal. +The declaration is a claim about **reading**, so it belongs to reading witnesses only. An authoring witness produces the artifact that creates an edge and never emits an observation to compare; it declares no coverage, and the absence is not an omission. + +A witness may only be compared on fields it declares. Declaring a field it cannot actually parse is a registration violation and grounds for removal. Conversely, an observation may not claim a class the registry does not carry for that witness: `semantic_surface_covered` must be a subset of the registered declaration, checked before the diff runs. ----- @@ -1066,10 +1160,11 @@ What the in-repo instance provides today: |Verdict and independence (Sections 6.3, 9.3)|`tools/ci/witness-verdict.py --registry research/witness-registry.json` | |Replay (Sections 8.4, 13) |`tools/ci/witness-verdict.py --compare-committed`; pinned by `tests/witness_verdict.rs` | |Registry (Section 5.3) |`research/witness-registry.json`; consistency enforced by `tests/witness_registry.rs` | +|Coverage declaration (Section 9.4) |`covers` per adopted reader in `research/witness-registry.json`; `tests/witness_registry.rs` holds every observation's `semantic_surface_covered` inside it| |CI gate (Section 10) |the `ifcopenshell-validate` job, observations and verdict published as build artifacts | |Machine-checkable schemas (Section 6)|`docs/schemas/witness-observation.schema.json`, `docs/schemas/witness-verdict.schema.json`| -What it does not provide, all of it umbrella scope (Section 5.3.1): containers (Section 10.3), hash chaining and the Ed25519 chain root (Section 12.2), the recording requirement (Section 12.3), the cross-platform determinism attestation (Section 9.5), exact version pinning of external witnesses (Section 9.6), and the `ci_eligible` / coverage-declaration registry fields. +What it does not provide, all of it umbrella scope (Section 5.3.1): containers (Section 10.3), hash chaining and the Ed25519 chain root (Section 12.2), the recording requirement (Section 12.3), the cross-platform determinism attestation (Section 9.5), exact version pinning of external witnesses (Section 9.6), and the `ci_eligible` registry field. The reference implementation is not normative; any conforming implementation may be used. Conformance is tested by validating observations and verdicts against the published schemas and by running the reference implementation's test suite against the candidate. @@ -1207,6 +1302,44 @@ Revit writes the complete fifteen-storey spatial hierarchy into an export carrying a single building element, so both edges are comparable here where `relations.IFCRELFILLSELEMENT` is comparable on only one. +## 20a. 1.1.1 (2026-08-30) + +Patch release, non-semantic (Section 16.1). No schema, diff-function, +canonicalizer, provenance or status-vocabulary change; every observation and +verdict valid under 1.1.0 is valid unchanged here. + +1. **§6.2 and §6.3 worked examples refreshed.** They quoted the + `magnetar-2024-core-interior` files as committed for 1.0.0 — two witnesses, + two lineages, an eight-field `entity_counts`-only surface — which 19b had + already labelled a dated snapshot. They are now regenerated from the + full-project artifact `magnetar-2024-core-interior-slim` at commit + `5da2570` (2026-08-30): three witnesses across three implementation + lineages, `schema_version: "1.1.0"` observations carrying `relations` and + `storeys`, and a thirteen-field verdict surface spanning all three field + classes with three categories excluded first-class. Trimming is confined to + four arrays and each trim is stated with its full length; every other byte + is the committed file. The `DISAGREE` and `REPLAY_DRIFT` blocks — which are + constructed, not committed — move to the same artifact so their identifiers + and the replay hash match the examples above them. The surrounding prose is + reconciled with what the new blocks show: the `schema_version` row of the + §6.2 key table, the `semantic_surface` row of the §6.3 key table (a surface + entry is `.`, not only `entity_counts.`), the §6 + preamble naming which artifact each subsection quotes, and the + load-bearing-properties paragraph, whose example of a type one witness + never emits was `IFCSHADINGDEVICE` and is now `IFCROOF` / `IFCBEAM` / + `IFCFLOWTERMINAL`. Closes rvt-rs #224. +2. **§5.3.1 coverage row updated from "not implemented" to implemented.** The + §9.4 coverage declaration is no longer umbrella scope: the reference + implementation's registry carries `covers` per adopted reader, drawn from + the §9.4 vocabulary, and `tests/witness_registry.rs` requires every + committed observation's `semantic_surface_covered` to be a subset of it. + §9.4 gains the two clarifications that implementation forced — the + declaration is reader-only, because an authoring witness emits an artifact + rather than an observation, and the subset rule is checked before the diff + runs — and §16.3 moves the row from "does not provide" to the provides + table, leaving `ci_eligible` as the only registry field still umbrella + scope. Closes rvt-rs #229. + ## 19b. 1.0.2 (2026-08-30) Patch release, non-semantic (Section 16.1). The §6.2 and §6.3 worked examples quote the observation and verdict as committed in the reference implementation on 2026-08-30. Later decoder work in that repository changes those files by design — the slab recovery in rvt-rs #212 moves `entity_counts.IFCSLAB` from 64 to 80, adds `IFCSHADINGDEVICE`, and takes `floors` off the excluded list — so the examples are now labelled dated snapshots rather than a live mirror of `research/witness/`. The protocol, the schemas, the diff function and the status vocabulary are unchanged; nothing here is normative. Readers wanting the current artifacts should read the committed files, which CI keeps in replay agreement with a fresh run. diff --git a/registry/witness-registry.schema.json b/registry/witness-registry.schema.json index 4c9e0e2..46190d7 100644 --- a/registry/witness-registry.schema.json +++ b/registry/witness-registry.schema.json @@ -34,11 +34,46 @@ "license": { "type": "string" }, "language": { "type": ["string", "null"] }, "status": { "type": "string", "enum": ["adopted", "candidate", "rejected"] }, + "covers": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "type": "string", + "enum": [ + "entity_counts", + "relations", + "storeys", + "layer_topology", + "linework", + "bounding_boxes", + "xdata_fields", + "text_content", + "3d_solids", + "meshes", + "bim_parameters" + ] + }, + "description": "OctetProof §9.4 coverage declaration: the semantic-surface classes this witness claims to read. Readers only — an authoring witness emits an artifact, never an observation, so it declares nothing. A witness is compared only on what it declares, and every committed observation's `semantic_surface_covered` must be a subset of this list." + }, "priority": { "type": "string", "enum": ["high", "medium", "low"] }, "lineage": { "type": "string", "description": "id of the witness this one is built on; independence (§9.3) counts them as one" }, "checked": { "type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$" }, "notes": { "type": "string" } - } + }, + "allOf": [ + { + "if": { "properties": { "kind": { "const": "author" } }, "required": ["kind"] }, + "then": { "not": { "required": ["covers"] } } + }, + { + "if": { + "properties": { "kind": { "const": "reader" }, "status": { "const": "adopted" } }, + "required": ["kind", "status"] + }, + "then": { "required": ["covers"] } + } + ] } }, "artifacts": { diff --git a/registry/witnesses.json b/registry/witnesses.json index 9612dc7..48db4fc 100644 --- a/registry/witnesses.json +++ b/registry/witnesses.json @@ -16,16 +16,16 @@ {"id": "las", "format": "ASPRS LAS/LAZ point cloud", "specification": "ASPRS LAS 1.4 published"} ], "witnesses": [ - {"id": "rvt-rs", "kind": "reader", "node": "rvt", "repo": "DrunkOnJava/rvt-rs", "license": "Apache-2.0", "language": "Rust", "status": "adopted", "notes": "the decoder under test; also writes ifc/gltf/svg"}, - {"id": "dwg-rs", "kind": "reader", "node": "dwg", "repo": "DrunkOnJava/dwg-rs", "license": "Apache-2.0", "language": "Rust", "status": "adopted", "notes": "sibling clean-room reader; 20.1% measured real-file entity coverage (R2018 sample 44.2%) — not yet a reliable witness for Revit exports; must parse 2D plan-view exports (LINE/LWPOLYLINE/ARC/CIRCLE/TEXT/MTEXT/INSERT/HATCH + LAYER/LTYPE/BLOCK) before the DWG edge is gated"}, - {"id": "autodesk-revit-exporter", "kind": "author", "node": "rvt", "repo": null, "license": "proprietary (licensed seat)", "language": null, "status": "adopted", "notes": "produces the canonical exports that create edges; only needed once per artifact"}, - {"id": "ifcopenshell", "kind": "reader", "node": "ifc", "repo": "IfcOpenShell/IfcOpenShell", "license": "LGPL-3.0", "language": "C++/Python", "status": "adopted", "notes": "runs as a separate process in CI; no linking, no copyleft contamination of the Rust workspace", "checked": "2026-08-30"}, + {"id": "rvt-rs", "kind": "reader", "node": "rvt", "repo": "DrunkOnJava/rvt-rs", "license": "Apache-2.0", "language": "Rust", "status": "adopted", "covers": ["entity_counts", "relations", "storeys"], "notes": "the decoder under test; also writes ifc/gltf/svg. Reproduced on both magnetar Core Interior edges: `entity_counts` from its own emitted STEP constructor histogram, `relations.IFCRELFILLSELEMENT` through the void/fill chain, `storeys.IFCBUILDINGSTOREY` with the elevation unit-normalized to feet"}, + {"id": "dwg-rs", "kind": "reader", "node": "dwg", "repo": "DrunkOnJava/dwg-rs", "license": "Apache-2.0", "language": "Rust", "status": "adopted", "covers": ["entity_counts"], "notes": "sibling clean-room reader; 20.1% measured real-file entity coverage (R2018 sample 44.2%) — not yet a reliable witness for Revit exports; must parse 2D plan-view exports (LINE/LWPOLYLINE/ARC/CIRCLE/TEXT/MTEXT/INSERT/HATCH + LAYER/LTYPE/BLOCK) before the DWG edge is gated. It has produced no observation yet — the rvt-to-dwg edge is `pending` and no DWG artifact exists — so its `covers` is the surface it would claim on that edge, not a reproduced agreement; `layer_topology`, `linework`, `bounding_boxes`, `xdata_fields` and `text_content` join only when the plan-view export lands"}, + {"id": "autodesk-revit-exporter", "kind": "author", "node": "rvt", "repo": null, "license": "proprietary (licensed seat)", "language": null, "status": "adopted", "notes": "produces the canonical exports that create edges; only needed once per artifact. Carries no `covers`: the §9.4 coverage declaration is a claim about what a witness *reads*, and an authoring witness never emits an observation to compare — its output is the artifact itself, hashed in `artifacts` and bound by an `edges` entry. `covers` is therefore reader-only by rule (tests/witness_registry.rs)"}, + {"id": "ifcopenshell", "kind": "reader", "node": "ifc", "repo": "IfcOpenShell/IfcOpenShell", "license": "LGPL-3.0", "language": "C++/Python", "status": "adopted", "covers": ["entity_counts", "relations", "storeys"], "notes": "runs as a separate process in CI; no linking, no copyleft contamination of the Rust workspace. Reproduced on both magnetar Core Interior edges via `by_type`, attribute access and `ifcopenshell.util.unit`", "checked": "2026-08-30"}, {"id": "libredwg", "kind": "reader", "node": "dwg", "repo": "LibreDWG/libredwg", "license": "GPL-3.0", "language": "C", "status": "candidate", "priority": "high", "notes": "~99% read coverage through R2018; 0.14 (2026-06-27) adds experimental R2010–R2018 write with CRC errors; usable as a CI process witness, never linked", "checked": "2026-08-30"}, {"id": "acadsharp", "kind": "reader", "node": "dwg", "repo": "DomCR/ACadSharp", "license": "MIT", "language": "C#", "status": "candidate", "priority": "high", "notes": "v3.7.1 (2026-08-18); ACIS payload reading for 3DSOLID/REGION/BODY since v3.6.51 (2026-07-29, PR #1139), MESH still unimplemented — the 3D-solid edge candidate; already consulted comments-only for the LZ77 offset ambiguity (dwg-rs CLEANROOM.md)", "checked": "2026-08-30"}, {"id": "ezdwg", "kind": "reader", "node": "dwg", "repo": "monozukuri-ai/ezdwg", "license": "MIT", "language": "Rust/Python", "status": "candidate", "priority": "medium", "checked": "2026-08-30", "notes": "Rust core with a Python API, R13–R2018, pushed 2026-08-23; independent implementation (not a LibreDWG binding)"}, {"id": "ezdxf", "kind": "reader", "node": "dxf", "repo": "mozman/ezdxf", "license": "MIT", "language": "Python", "status": "candidate", "priority": "high", "notes": "DXF is the documented twin of DWG; Revit exports DXF too", "checked": "2026-08-30"}, {"id": "dxf-rs", "kind": "reader", "node": "dxf", "repo": "ixmilia/dxf-rs", "license": "MIT", "language": "Rust", "status": "candidate", "priority": "medium", "checked": "2026-08-30"}, - {"id": "ifc-lite", "kind": "reader", "node": "ifc", "repo": "LTplus-AG/ifc-lite", "license": "MPL-2.0", "language": "Rust", "status": "adopted", "version": "7.1.1", "crate": "ifc-lite-core", "checked": "2026-08-30", "notes": "third independent IFC reading witness. Verified 2026-08-30 against crates.io and the GitHub API: crate `ifc-lite-core` 7.1.1 published 2026-08-27, license MPL-2.0 in both the crate metadata and the repository LICENSE — an earlier entry here recorded `MIT` and a bare `ifc-lite` repo slug; both were wrong and are corrected. Independent implementation: a hand-rolled byte-level STEP scanner (src/parser/scanner.rs) plus a nom tokenizer, no IfcOpenShell code and no binding to it — the project *verifies its geometry against* IfcOpenShell, which is validation, not lineage. Adopted as a separate CI process via tools/ci/witness-ifc-lite, its own workspace root so MPL code is never linked into the Apache-2.0 tree; pinned `ifc-lite-core = \"=7.1.1\"` per §9.6."}, + {"id": "ifc-lite", "kind": "reader", "node": "ifc", "repo": "LTplus-AG/ifc-lite", "license": "MPL-2.0", "language": "Rust", "status": "adopted", "version": "7.1.1", "crate": "ifc-lite-core", "covers": ["entity_counts", "relations", "storeys"], "checked": "2026-08-30", "notes": "third independent IFC reading witness. Reproduced on both magnetar Core Interior edges via `EntityScanner`, `parse_entity` and `ifc_lite_core::extract_length_unit_scale`. Verified 2026-08-30 against crates.io and the GitHub API: crate `ifc-lite-core` 7.1.1 published 2026-08-27, license MPL-2.0 in both the crate metadata and the repository LICENSE — an earlier entry here recorded `MIT` and a bare `ifc-lite` repo slug; both were wrong and are corrected. Independent implementation: a hand-rolled byte-level STEP scanner (src/parser/scanner.rs) plus a nom tokenizer, no IfcOpenShell code and no binding to it — the project *verifies its geometry against* IfcOpenShell, which is validation, not lineage. Adopted as a separate CI process via tools/ci/witness-ifc-lite, its own workspace root so MPL code is never linked into the Apache-2.0 tree; pinned `ifc-lite-core = \"=7.1.1\"` per §9.6."}, {"id": "web-ifc", "kind": "reader", "node": "ifc", "repo": "ThatOpen/engine_web-ifc", "license": "MPL-2.0", "language": "C++/WASM", "status": "candidate", "priority": "medium", "checked": "2026-08-30", "notes": "0.77 (2026-03-06); browser/WASM parser, independent of IfcOpenShell"}, {"id": "stepcode", "kind": "reader", "node": "ifc", "repo": "stepcode/stepcode", "license": "BSD-3-Clause (NOASSERTION on GitHub)", "language": "C++", "status": "candidate", "priority": "low", "notes": "generic Part 21 parser — syntax-level witness only", "checked": "2026-08-30"}, {"id": "gltf-rs", "kind": "reader", "node": "gltf", "repo": "gltf-rs/gltf", "license": "MIT/Apache-2.0", "language": "Rust", "status": "candidate", "priority": "medium", "notes": "witness for rvt-rs glTF output validity"}, @@ -95,6 +95,7 @@ "tolerance_changes": "reviewed change with a written reason in the manifest", "adopting_a_witness": "run it as a separate process in CI; never link copyleft or non-Rust code into the workspace", "independence": "two witnesses count as independent only if neither is built on the other's implementation (uncad→LibreDWG, FreeCAD BIM→IfcOpenShell, GDAL DGN→dgnlib, Ara3D mesh→web-ifc are the same witness as their base)", - "candidate_claims": "coverage or pass-rate figures quoted in a candidate's notes are the project's own claims until an agreement here reproduces them" + "candidate_claims": "coverage or pass-rate figures quoted in a candidate's notes are the project's own claims until an agreement here reproduces them", + "coverage_declaration": "every adopted reader declares `covers`, the OctetProof §9.4 semantic-surface classes it claims — the registry-level counterpart of an observation's `semantic_surface_covered`. An authoring witness never declares it: coverage is a claim about reading, and an author emits an artifact rather than an observation. A witness is compared only on what it declares, so every committed observation's `semantic_surface_covered` must be a subset of its witness's `covers` (enforced by tests/witness_registry.rs); declaring a class the witness cannot parse is a registration violation" } } diff --git a/tools/check-registry.py b/tools/check-registry.py index d25aff9..ab30e74 100755 --- a/tools/check-registry.py +++ b/tools/check-registry.py @@ -101,6 +101,13 @@ def main() -> int: witnesses = {w["id"]: w for w in registry.get("witnesses", [])} edges = {e["id"] for e in registry.get("edges", [])} + for wid, entry in sorted(witnesses.items()): + covers = entry.get("covers") + if entry.get("kind") == "reader" and entry.get("status") == "adopted" and not covers: + errors.append(f"registry: adopted reader {wid} declares no `covers` (spec 1.1.1 section 9.4)") + if entry.get("kind") == "author" and covers: + errors.append(f"registry: author {wid} must not declare `covers` (coverage is a claim about reading)") + for w in registry.get("witnesses", []): if w["node"] not in nodes: errors.append(f"witness {w['id']}: node {w['node']!r} is not declared") @@ -147,6 +154,15 @@ def main() -> int: f"{label}: {wid} manifest says registry_status {declared_status!r}, " f"registry says {entry.get('status')!r}" ) + for obs_path in sorted((manifest_path.parent / "observations").glob("*.json")): + obs = json.loads(obs_path.read_text()) + wid = obs.get("witness_id") or obs_path.stem + entry = witnesses.get(wid) + if entry is None: + continue + extra = sorted(set(obs.get("semantic_surface_covered") or []) - set(entry.get("covers") or [])) + if extra: + errors.append(f"{label}: {wid} observation covers {extra} beyond its registry `covers` declaration") authoring = manifest.get("bridge", {}).get("authoring_witness") if authoring and authoring not in witnesses: errors.append(f"{label}: authoring_witness {authoring!r} is not in the registry") diff --git a/tools/jsonschema_mini.py b/tools/jsonschema_mini.py index 2dba4cd..03ea71f 100755 --- a/tools/jsonschema_mini.py +++ b/tools/jsonschema_mini.py @@ -11,7 +11,7 @@ Implemented keywords: `type` (including union types and `null`), `required`, `properties`, `additionalProperties` (schema form and `false`), `items`, `enum`, `const`, `pattern`, `minLength`, `minItems`, `maxItems`, -`uniqueItems`, `minimum`. Annotation-only keywords (`$schema`, `$id`, `title`, +`uniqueItems`, `minimum`, `maxItems`, `allOf`, `if`/`then`/`else`, `not`. Annotation-only keywords (`$schema`, `$id`, `title`, `description`) are ignored. `maxItems` arrived with OctetProof 1.1.0: the observation schema's `relations` @@ -35,6 +35,7 @@ ANNOTATION = {"$schema", "$id", "title", "description", "$comment", "examples", "default"} IMPLEMENTED = { "type", "required", "properties", "additionalProperties", "items", + "allOf", "if", "then", "else", "not", "enum", "const", "pattern", "minLength", "minItems", "maxItems", "uniqueItems", "minimum", } @@ -79,6 +80,22 @@ def check(node, schema, path: str, errors: list[str]) -> None: if node is None: return + if "allOf" in schema: + for i, sub in enumerate(schema["allOf"]): + check(node, sub, f"{path}(allOf[{i}])", errors) + if "not" in schema: + scratch: list[str] = [] + check(node, schema["not"], path, scratch) + if not scratch: + errors.append(f"{path}: matches a schema it must not match (`not`)") + if "if" in schema: + scratch = [] + check(node, schema["if"], path, scratch) + if not scratch: + if "then" in schema: + check(node, schema["then"], f"{path}(then)", errors) + elif "else" in schema: + check(node, schema["else"], f"{path}(else)", errors) if "const" in schema and node != schema["const"]: errors.append(f"{path}: expected const {schema['const']!r}, got {node!r}") if "enum" in schema and node not in schema["enum"]: