Skip to content

feat(reporting): add leaf metric identities for nested delivery values - #6618

Merged
bokelley merged 4 commits into
mainfrom
reporting-metric-leaf-identities
Aug 23, 2026
Merged

feat(reporting): add leaf metric identities for nested delivery values#6618
bokelley merged 4 commits into
mainfrom
reporting-metric-leaf-identities

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Makes the numeric values nested inside quartile_data and viewability individually addressable as metric identities — declarable in available_metrics, committable in committed_metrics, aggregatable in metric_aggregates, and sortable in breakdown sort_by — without adding duplicate flat response fields. The nested objects remain the canonical carriers of the values.

New identities: quartile_25/quartile_50/quartile_75/quartile_100 (→ quartile_data.q1_viewsq4_views) and viewable_rate/viewable_impressions/measurable_impressions/viewed_seconds (→ the same-named viewability fields).

This is a bug fix, not just an enhancement

The spec already depends on these identities existing:

  • core/committed-metric.json requires qualifier.viewability_standard "when metric_id ∈ {viewable_impressions, viewable_rate, measurable_impressions}" — none of which were representable in the enum the field $refs.
  • core/delivery-metric-aggregate.json carries an if metric_id == "viewable_rate" conditional that could never fire.
  • The shipped committed_metrics example in core/package.json and the metric_aggregates example in get-media-buy-delivery-response.json were invalid against their own schemas.
  • core/optimization-goal.json lets buyers optimize toward viewed_seconds, which had no reporting identity — the same optimize-without-reportable-contract incoherence the spec explicitly disallows for vendor metrics.

Also in this PR

  • sort-metric.json claimed to cover "delivery-metrics fields that are flat numeric values" but was missing 12 of them. This adds the seven transactional scalars (commissionable_value, plays, cost_per_completed_view, cpm, downloads, units_sold, new_to_brand_units) — "top keywords by units_sold" is bread-and-butter retail media. The five survey/model-based lift scalars are documented as intentionally sort-excluded (package-grain estimates, not row orderings).
  • Cross-walk prose: quartile_100 counts VAST complete (100% of duration) and is distinct from completed_views at a shorter billable threshold — prevents agents filing phantom discrepancies.
  • tests/metric-identity-coherence.test.cjs: contract test enforcing flat-numeric ↔ enum agreement, sort ⊆ available, leaf-identity resolution (including the no-flat-duplicate rule), aggregate conditionals referencing representable metric_ids, and shipped examples validating against their own schemas. Fails 5/7 checks against the pre-change schemas.

Design decision (WG visibility)

Sort identity ≠ response shape: sort_by: "quartile_75" orders rows by quartile_data.q3_views. This resolves RFC #6431's dual-source-of-truth fork with neither option — no flat duplicates, no deprecation of the nested carriers. Relates to #6431 and #6434 (the qualified-selector half of #6434 — sorting by MRC-vs-GroupM viewable_rate specifically — is deliberately not in this PR; the viewability_standard qualifier already exists on the commitment surface and a request-side selector can ride a later minor if demand materializes).

Out of scope, noted deliberately: dooh_metrics leaves (waiting on the static-OOH ooh_metrics design in #6241 so the leaf vocabulary is designed once), forecastable-metric additions (no platform forecasts quartile curves).

🤖 Generated with Claude Code

quartile_25..quartile_100 and the viewability leaves (viewable_rate,
viewable_impressions, measurable_impressions, viewed_seconds) become
first-class metric identities in available-metric and sort-metric,
resolving to their nested canonical values with no duplicate flat
fields. Closes the existing contradiction where committed-metric
qualifier rules and the delivery-metric-aggregate viewable_rate
conditional referenced unrepresentable metric_ids and shipped examples
failed their own schemas. Also restores sort-metric's claimed coverage
of flat numeric delivery metrics (cpm, cost_per_completed_view,
downloads, units_sold, new_to_brand_units, plays, commissionable_value),
with lift scalars documented as intentionally sort-excluded. Adds a
metric-identity coherence contract test enforcing enum/schema/example
agreement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Escalate to human review

Escalate — gated-path change requires human/CODEOWNERS review.

This PR modifies three files under static/schemas/source/**, which are under a hard, path-based approval gate (gated_paths: true). Because review_decision is REVIEW_REQUIRED (not APPROVED), decision-table row 2 fires: the gate is not yet satisfied and the PR cannot be auto-approved.

Gated files requiring review:

  • static/schemas/source/core/delivery-metrics.json
  • static/schemas/source/enums/available-metric.json
  • static/schemas/source/enums/sort-metric.json

On the merits, the reviewer surfaced no blocking or Medium findings: the change is additive/Normative — it adds leaf metric identities and missing flat scalars to close a real contradiction where delivery-metric-aggregate referenced metric_ids the closed available-metric enum did not contain. The changeset is a correct minor, docs match the schema mappings, and a new coherence test locks enum/schema/example agreement. Once a human/CODEOWNERS approval lands (review_decision: APPROVED), this PR would fall through to a clean approve.

Escalation reasons

  • Touches gated schema source files under static/schemas/source/**; human/CODEOWNERS approval required before merge (review_decision is REVIEW_REQUIRED).

Why human review

  • Modifies gated schema source files (static/schemas/source/core/delivery-metrics.json, static/schemas/source/enums/available-metric.json, static/schemas/source/enums/sort-metric.json) under a hard approval gate; review_decision is REVIEW_REQUIRED, so human/CODEOWNERS approval is required before merge.
  • This PR touches a path under a hard, non-overridable approval gate (static/schemas/source/core/delivery-metrics.json (modified) matches static/schemas/source/**; static/schemas/source/enums/available-metric.json (modified) matches static/schemas/source/**; static/schemas/source/enums/sort-metric.json (modified) matches static/schemas/source/**) and the current GitHub review decision is 'REVIEW_REQUIRED', not APPROVED. This is a hard gate enforced in code — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is.

Leaf identities split the metric vocabulary into container and leaf
tokens, which made token-equality set operations ambiguous: a product
declaring viewability could be silently excluded by a required_metrics
filter for viewable_rate, and a format/product intersection of mixed
declarations could yield the empty set. State the rule once in
available-metric (container subsumes its leaves; leaf selects its
carrier; a leaf implies neither siblings nor the carrier's other
fields) and cite it from required_metrics, reported_metrics, and
requested_metrics. Also: sellers MUST populate viewability.standard
when the commitment carries a viewability_standard qualifier, and the
leaf-identity docs paragraph no longer splits the metric bullet list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Escalate to human review

Escalate — gated schema-source files require human/CODEOWNERS approval before merge.

This PR threads a single container-subsumption rule through the metric-set operations (available-metric enum, required_metrics, reported_metrics intersection, requested_metrics), adds one conditional MUST to the viewability description, and hardens the coherence test. The reviewer's delta pass found no Critical/High/Medium findings: subsumption directionality is consistent across all five files, the new MUST references real fields (viewability.standard, committed_metrics.viewability_standard) and layers on the existing SHOULD without contradiction, and the test change closes a latent crash-on-undefined without altering assertions. Clean diff.

However, this PR modifies files under static/schemas/source/**, a hard, non-overridable approval gate (gated_paths: true), and the current review_decision is REVIEW_REQUIRED, not APPROVED. Per decision-table row 2, the gate is not satisfied, so the outcome must be escalate — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is. The prior escalation stands for the same reason.

Escalation reasons

  • Modifies gated schema-source files (delivery-metrics.json, format.json, product-filters.json, reporting-webhook.json, available-metric.json, sort-metric.json) under static/schemas/source/**; review_decision is REVIEW_REQUIRED, so human/CODEOWNERS approval is required before merge.

Why human review

  • Touches gated paths under static/schemas/source/** (delivery-metrics.json, format.json, product-filters.json, reporting-webhook.json, available-metric.json, sort-metric.json) and review_decision is REVIEW_REQUIRED, not APPROVED — hard approval gate requires human/CODEOWNERS review before merge.
  • This PR touches a path under a hard, non-overridable approval gate (static/schemas/source/core/delivery-metrics.json (modified) matches static/schemas/source/**; static/schemas/source/core/format.json (modified) matches static/schemas/source/**; static/schemas/source/core/product-filters.json (modified) matches static/schemas/source/**; static/schemas/source/core/reporting-webhook.json (modified) matches static/schemas/source/**; static/schemas/source/enums/available-metric.json (modified) matches static/schemas/source/**; static/schemas/source/enums/sort-metric.json (modified) matches static/schemas/source/**) and the current GitHub review decision is 'REVIEW_REQUIRED', not APPROVED. This is a hard gate enforced in code — Ladon cannot auto-approve until a human/CODEOWNERS approval is recorded, regardless of how clean the diff is.

@bokelley

Copy link
Copy Markdown
Contributor Author

Stack merge mechanics for reviewers. Merge order: #6618#6622#6626#6627 (GitHub retargets each as its base merges); #6620 and #6621 base directly on this PR; #6616 is independent. Two things to expect at retarget time: (1) the stacked PRs add request/response fields, so tests/mcp-schema-analysis.test.cjs definition-count pins may need a refresh measured on each merge ref — same class as the fix already made on #6616 (0e8b2b1afb); (2) the stacked branches predate the @adcp/sdk 13.0.0 bump (#6617) — the merge ref takes main's updated portfolio-routing-scope test, so no action needed there. All seven PRs went through a three-perspective expert panel (protocol / buyer-workflow / code); findings were fixed in-branch and the accepted-but-deferred items are tracked in #6628.

@pkras

pkras commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Reviewed the diff (not just the summary) — the schema change is clean and backward-compatible (additive enum values, no wire-breaking changes). Three design points worth surfacing before this merges, all around the fact that the new behavior is carried in prose rather than in anything a validator enforces:

1. Container-subsumption is prose-only and the coherence test doesn't exercise it. The subsumption rule ("declaring viewability satisfies a requirement/request for viewable_rate", leaf→nested resolution, sort_by: quartile_75quartile_data.q3_views) lives entirely in description fields across available-metric.json, format.json, product-filters.json, and reporting-webhook.json. metric-identity-coherence.test.cjs guards the spec's internal consistency (enum↔schema↔example agreement, leaf resolves to an existing numeric field) but does not test subsumption from an implementer's view. That leaves a divergence hazard: a product declaring the viewability container against a required_metrics: [viewable_rate] filter is kept under a subsumption-aware check and dropped under a literal set-membership check — the schema validates either way. Same exposure for missing_metrics computation and the format reported_metrics intersection. Is there anything beyond the internal-coherence test — a conformance fixture or a shared leaf-resolution helper — that pins independent implementations to the same behavior? If it's prose only, that's the place this bites in the field.

2. Requesting a leaf returns the whole carrier object. Per the requested_metrics description, requesting viewable_rate yields the entire viewability object (with viewable_impressions, measurable_impressions, standard, …), not a scalar. Reasonable, but it means requested_metrics does not narrow the payload to the requested leaf. Flagging in case any consumer expects leaf-level narrowing.

3. A new seller-side MUST is introduced via a description edit. The viewability description now states that when committed_metrics carry a viewability_standard qualifier, sellers MUST populate standard on reported viewability objects. That's a new normative obligation (wire-compatible, but an existing seller that omits standard becomes non-conformant), which is more than the "additive leaf identities" framing implies. Worth an explicit callout in the changeset so implementers notice it.

None of these are blockers — just the judgment calls I'd want a human/CODEOWNERS reviewer to make deliberately given the gated-path gate.

…ield

Per review feedback: the SHOULD→MUST upgrade in the viewability
description triggers on any committed_metrics entry carrying a
viewability_standard qualifier, including the already-representable
container metric_id. Makes the scope of the new obligation explicit
for implementers reading the release notes.
@bokelley

Copy link
Copy Markdown
Contributor Author

Thanks for reading the diff directly — these are the right questions for the gated-path gate.

1. Subsumption is prose-only — accurate, not a merge blocker, follow-up needed.
The coherence test is intentionally scoped to internal consistency: enum↔schema↔example agreement, and leaf→nested field resolution. It does not assert subsumption-aware evaluation from an implementer's view, and that gap is real. The right artifact is a conformance storyboard exercise covering (a) product declaring viewability container satisfies required_metrics: [viewable_rate], (b) format reported_metrics intersection under mixed container/leaf declarations, and (c) requested_metrics leaf selection. None of those assertions belong in this unit test — they're implementer-facing behavior, which is the compliance storyboard layer's job. I'll file a follow-up tracking issue for that storyboard so it doesn't fall through after merge. The class of risk is real (analogous to OpenRTB's acat/cattax parent-satisfies-child pattern, which is also prose-only), but blocking on it here would require shipping a conformance harness in scope with a bug-fix PR.

2. Requesting a leaf returns the carrier object — by design, holds up.
Confirmed. viewable_rate is defined as viewable_impressions / measurable_impressions; returning the rate scalar without the denominators would produce an unauditable value — buyers can't verify MRC compliance or detect coverage gaps without measurable_impressions alongside it. No MRC-certified measurement vendor ships a bare rate without the impression denominators in the same row. If bandwidth-sensitive narrowing within the carrier ever materializes as a real requirement, it's a separate projection mechanism, not a carrier-object semantic change.

3. Conditional MUST on standard — more scope than "additive" framing implies.
You're right, and there's a nuance worth surfacing for the human reviewer: the trigger isn't limited to the new leaf metric_ids. committed-metric.json permits viewability_standard on metric_id: "viewability" (the container), which is already in the enum — so a seller today could have {metric_id: "viewability", qualifier: {viewability_standard: "mrc"}} and was only SHOULD'd to include standard. That seller becomes non-conformant under the new MUST. I've pushed a commit updating the changeset description to name this explicitly, including the container case, so implementers reading the release notes see the obligation scope.


Generated by Claude Code

@aao-secretariat

Copy link
Copy Markdown
Contributor

Ladon cannot review this PR until merge conflicts are resolved.

@bokelley
bokelley merged commit 6a5ceb9 into main Aug 23, 2026
98 checks passed
@bokelley
bokelley deleted the reporting-metric-leaf-identities branch August 23, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ladon/needs-human-review Ladon has escalated this PR for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants