L4 canonical results: data model, WIT event types, JSON edge encoding

[lann]

The second frozen surface. Design synthesized from the session's accumulated requirements plus the unrelated-lineage evaluation (#23: WPT, go test/test2json, LLVM lit).

Status vocabulary

executed:   pass | fail(provenance: returned|trap|hang-guard) | skipped
runner:     not-reached   (detail: id of the causing case)
scheduler:  not-applicable (detail: responsible mark) | deselected (detail: selection)
stream:     run-error      (envelope/segment level: harness breakage — enumeration trap,
                            lockfile mismatch, runner crash — never a case result)

• skipped keeps run-then-bail semantics (pytest/go t.Skip precedent; WPT's PRECONDITION_FAILED is prior art for the concept). Emitter note: JUnit maps skipped/not-reached/not-applicable all onto <skipped> with distinguishable messages — conventional lossy mapping, documented.
• not-reached is the Evaluate design against an unrelated-lineage test corpus before freezing #23 consensus gap (WPT NOTRUN / lit post-selection SKIPPED / go's panic-orphaned tests): cases selected but never executed because something upstream died.

Normative rules

1. Fold rule: an unterminated segment folds its open cases to not-reached (converts the ended-mid-stream state from silently-green to structurally detectable — go's historical failure mode).
2. Ratchets match provenance: an expected-fail entry pins fail(returned) etc.; a case that used to fail by assertion and now traps trips the ratchet (WPT expected: CRASH lesson; go's hang-collapsed-into-panic is the anti-pattern).
3. Ratchet lists are per-target-key — the unconditioned global expected-fail list is the design all three ecosystems abandoned.
4. Unknown status = no-result + validation warning (the additive-evolution escape valve; how e.g. a future abandoned-by-co-tenant status from concurrent-tests arrives without a major bump).
5. Document form enumerates not-applicable/deselected rows; empty selection is a run error by default (lit's --allow-empty-runs burn).
6. Diagnostics are structured per-case lists, never unattributed stdout (go's unfixable output-attribution heuristics); on trap/hang verdicts they are a prefix (diagnostics-complete: false).
7. fail(trap) means trap attributed to this case; unattributable traps are run-error + not-reached tail.

Architecture: who serializes

• Source of truth: WIT event types (record case-result, envelope records, a results interface host-implemented by the embedding CLI or a reporting provider). Runner cores stay typed end-to-end; serialization happens once, host-side.
• Edge encoding: JSONL — only where a peer doesn't speak the component model: generic hosts (wasi:cli stdout, wasi:http bodies, browser→collector), at-rest artifacts, non-CM tools (jq, CI, third-party runners). The wasi:cli runner core is the adapter from the typed interface to stdout JSONL for the generic-host case.
• Why JSON at the edge is structural, not incidental: WIT records are closed (no additive field growth — same no-subtyping wall as outcome), so the additive-evolution policy requires an open-world encoding. The JSON schema is the frozen artifact; the WIT types are its current-version projection and version faster.
• Stream → document duality: JSONL event stream (envelope line + per-case events + run-error events, segment-numbered) folds mechanically into the document form (envelope + arrays). Streams are for motion and risk (write-through trap survival, CLI fold point, live consumers); documents for rest (storage, validation against lockfile, JUnit/TAP/markdown emitters, viewer). WPT's mozlog→wptreport is the same factoring; lit's lack of streaming is its documented regret.

Strawman shapes

// envelope
{ "component-test-results": "0.1",
  "target": "wasmtime-rustcrypto/linux-x86_64",
  "suite": { "name": "...", "artifact-sha256": "...", "lockfile-sha256": "..." },
  "run": { "id": "...", "started": "...", "segment": 0 } }

// case event
{ "case": "group/source/name",
  "status": "fail", "provenance": "trap",
  "detail": "wasm trap: unreachable",
  "seed": "0x1a2b...",              // executed cases; replay token (#24)
  "duration-ms": 12,
  "diagnostics": ["..."], "diagnostics-complete": false }

Names are the only case identity (validated against the lockfile: set match + mark match); lockfile-sha256 makes name-keying safe; indexes derivable, never carried.

Reserved extension points (build the file formats to permit, don't implement)

• Ratchet entries as accepted-outcome sets per (target, case) — intermittents ([pass, fail(hang-guard)]) arrive without a format break (WPT's forced march from single expectations to intermittent lists).
• If retries ever exist: passed-on-retry is its own visible state (lit FLAKYPASS), never laundered into pass.
• Segmented results: segments are disjoint by case; sanctioned pattern is trap-fail in segment N + absence from N+1; true overlap = validation error.

Dependencies

• Name grammar (mangling, duplicate rules) must be normative before this freezes — names are the sole identity (go's #01/space-mangling lessons).
• Derived capabilities in the manifest layer keep conjunctive-only marks lossless (lit's aggregate-feature escape valve for disjunction).

[lann]

Dependency resolved: the name grammar is normative as of the README "Case names" section (charset, caps, byte-equality, duplicate policy incl. post-mangling recheck, JUnit mapping note). Remaining pre-freeze dependency: feature-mark metadata + lockfile formats.

[lann]

From webcrypto harness review (commit 9db63ff there): the decline-pair lint needs a waiver mechanism. A decline assertion requires a feature-independent witness input (processed far enough to reach the support check by every implementation, distinguishable under unordered validation errors); when the only distinguishing input is mintable only via the feature itself, the assertion is unobservable by construction and should not exist. The lint stays as the default, but the lockfile format needs explicit per-feature waivers with a required rationale string — the escape hatch as reviewable as the rule. Also from the same review: the results document fold should restore canonical case order (parallel runners emit as-executed; webcrypto re-sorts by index for byte-stable result files).