Report metric bounds, uncertainty and ids the way each harness computed them by borgr · Pull Request #246 · evaleval/every_eval_ever

borgr · 2026-08-10T16:40:00Z

Stacked on #245 (fix/converter-metric-naming-3fields), which is itself stacked on #244. Review those first; diff against the naming branch, not main.

What / source

The three harness converters were each asserting things about their metrics that the upstream logs never say. This PR makes each converter report what its harness actually computed, and say "unknown" where the harness does not know.

Four groups of change, all funnelling through one new module (converters/common/metrics.py) so a rule is stated once for every converter:

1. Bounds and direction, only where they are known. Every HELM and Inspect metric was published as continuous on [0, 1] with lower_is_better: false, so a perplexity, a standard deviation and an accuracy all came out looking like accuracies. Now a metric's range comes from its name, layered per harness — lm-eval's bleu is sacrebleu's 0–100 while HELM's bleu_1 is nltk's sentence_bleu 0–1, so a bare name cannot carry a range. A metric in no table gets no bounds and an additional_details.bounds_status: unknown marker, because min_score/max_score are nullable and "not provided" is true, while [0, 1] on an unbounded metric is not. Each record counts its unknown-bounds metrics in source_metadata.additional_details, so the gap is visible without reopening every file — and test_unknown_bounds_are_counted_in_the_source_metadata derives that count from the published results for every case, which is how the Inspect converter was caught reporting null while publishing an unbounded metric.

Dispersion metrics (std, stderr, bootstrap_stderr, var) get polarity: not_applicable: lower_is_better is a required boolean with no "does not apply" value, and a standard deviation is not better when small.

One of the ranges this PR introduced was itself wrong and is fixed here: brier_score was declared [0, 1], the two-class definition. lm-eval computes mean(sum((softmax(loglikelihoods) - one_hot(gold)) ** 2)) (api/task.py), which over n classes reaches 2.0 when all the mass sits on one wrong class. Since validation_core.py warns on a score outside its declared range and the converter suite requires zero warnings, the too-narrow bound would have failed the merge gate on a legitimate score. Now [0, 2], pinned by test_a_multi_class_brier_score_is_allowed_its_full_range.

2. Sample counts that count samples. HELM took num_samples from a stat's count, which is its train-trial count — 1 for nearly every published run, whatever the instance count. It now counts per-instance stats keyed per split and perturbation, falls back to HELM's own per-split num_instances for the worst-case perturbation stats that have none, and moves the trial count to score_details.details.num_train_trials. Inspect took num_samples from the length of a sample list that a header-only log does not populate; it now reads the results header.

3. Uncertainty, as each harness computed it — and never dropped. HELM republished the 0.0 spread over a single train trial as a measured standard_deviation; the schema defines that field over per-sample scores, so it is now omitted rather than reported as zero. Inspect dropped a standard error of exactly 0.0 as if it were absent (falsy, not missing), routed nothing for bootstrap_stderr, and emitted a scorer's std as a score of its own alongside the score it describes — it now carries it as uncertainty.standard_deviation on those scores, unless std is all that scorer reported. lm-eval derives the standard-error method from the aggregation its log records instead of asserting 'bootstrap', and reports a resample count only for aggregations that are actually resampled.

bootstrap_stderr was in neither the shared bounds nor DISPERSION_METRICS while stderr was in both, so a converter publishing the former claimed an unknown range and an applicable direction. It is a standard error computed by resampling rather than in closed form — same bounds, same polarity.

4. metric_id, so a consumer can join across harnesses. The previous PR gave every result a metric_name; nothing tied lm-eval's exact_match to HELM's. metric_config_fields now resolves a metric name to the eval-card-registry's canonical slug where the registry carries one (18 of 45 names), and to <harness>.<name> where it does not — marked metric_id_status: unregistered, so a namespaced id claims a stable join key within the harness and no global identity. Both forms record the registry revision they were resolved against (8b83e9c, that repo's main), because an unregistered marker with no revision beside it does not tell a reader whether the entry has since been added.

metric_kind (the family a metric aggregates safely within) and metric_unit come along because they answer what an id alone leaves open. The unit is derived from the resolved bounds, not listed: lm-eval's sacrebleu bleu reports percent and HELM's nltk bleu_1 reports proportion from their bounds alone, with neither spelled out.

Why the map is hand-resolved, and how you check it

A converter has to run offline and give the same answer on every run, so CANONICAL_METRIC_IDS is 18 hand-verified entries against a pinned revision rather than a network call to the registry's resolver. tools/verify_metric_ids.py re-does the resolution against a registry checkout and fails (exit 1) on the three ways the map can rot:

$ uv run python -m tools.verify_metric_ids --seed ../eval-card-registry/seed/metrics.yaml
ids resolved against revision 8b83e9c
18 mapped, 45 names checked

MAPPED ID NO LONGER IN THE REGISTRY: 0
NOW RESOLVABLE, STILL NAMESPACED: 0
AMBIGUOUS IN THE REGISTRY: 0

NAMESPACED, WANTING A REGISTRY ENTRY: 22
  acc_norm, bits_per_byte, brier_score, byte_perplexity, calibration_error,
  chain_of_thought_correctness, chrf, classification_macro_f1,
  classification_micro_f1, ece, ifeval_strict_accuracy, math_equiv,
  math_equiv_chain_of_thought, mc1, mc2, mcc, prefix_exact_match,
  quasi_exact_match, quasi_prefix_exact_match, rougeLsum, ter, word_perplexity

HARNESS SLUGS: lm-evaluation-harness ok, helm ok, inspect_ai NOT IN REGISTRY

That last list is not a failure — it is the content of a follow-up eval-card-registry PR, together with an @k slug family for HELM's best-of-k metrics and an inspect_ai harness entry. It is a tool rather than a test because it needs a sibling repo checkout, which CI does not have.

Two matching rules in it are worth a second of your attention: normalization keeps +, so chrF++ never answers to chrf; and chrf (plain) is in the gap list because the registry's chrf-plus-plus is a different metric.

Interaction with #222. That PR adds check_metric_identity, which warns on a missing metric_id, a semantically-empty one (score, mean, overall, …) and one equal to evaluation_name. I ran that branch's validator over all six files these three converters publish: 6 files, 0 complaints, exit 0, then set one metric_id to 'score' as a tripwire and confirmed the check fires. Namespaced ids survive because _normalize_metric_id maps -→_ but leaves . alone, so inspect_ai.mean is never generic. Since all three converters now always set an id, this PR removes #222's "missing" finding for them.

Checklist

uv run python -m every_eval_ever validate <files> clean, no warnings, at the final data/<collection>/<dev>/<model>/ path — 6 files, 0 complaints, exit 0; asserted by the stack's harness on every run
every unconvertible source row is in adapter_reports/ with a non-zero exit — and one case improves: a HELM run whose stats are all bookkeeping is now reported per run instead of failing the whole invocation from inside the publisher
offline unit test added + full uv run pytest tests green — core 438 passed / 39 skipped, full 524 passed / 1 skipped
uv run ruff check clean
model/benchmark/metric ids resolve in the registry — 18 resolve against 8b83e9c; the other 22 are namespaced, marked unregistered, and listed above for the registry PR. tests/test_converter_conversion.py::test_every_metric_id_is_either_canonical_or_openly_namespaced is the enforcement: an id is either in the verified map or carries its harness prefix, never a bare unqualified guess.
content spot-checked — no answer leakage, nothing double-counted, evaluation_id unaffected. tests/test_converter_metric_bounds.py (16 tests) pins the shared rules directly.

Review lane

Fast
Needs a human — a material change in outcome. min_score/max_score/score_type disappear from metrics whose range is unknown, num_samples changes value for HELM, a HELM standard_deviation disappears, an Inspect std stops being a score and becomes an uncertainty field, and every result gains a metric_id. Same scores, different metadata about them.

Design agreed in: not yet. The specific calls I would want a maintainer to confirm, in order of how much they would cost to change later:

Namespaced ids as the fallback. helm.quasi_exact_match is data that a later registry entry supersedes — every one of those 22 becomes a canonical id in a follow-up, and any record already published under the namespaced form then needs remapping. The alternative is a null metric_id until the registry catches up, which is honest but makes the field useless for the harnesses' most common metrics.
Omitting bounds rather than defaulting to [0, 1]. This is the AGENTS.md "report, don't interpret" principle applied literally. Of the 45 names the tables carry, 41 get a range and 4 deliberately do not (calibration_error, cer, ece, wer — an error rate with no upper bound in the general case); any metric outside every table also gets none, where before HELM and Inspect stamped [0, 1] on it. On these fixtures that is 1 of 28 results (Inspect's mean). If any downstream consumer treats a null max_score as an error rather than as unknown, they will feel it there first.
An Inspect std is no longer a score. It is the one place where a result count changes for a reason other than naming.

Decisions & coverage

Decision / where: a metric absent from every bounds table gets no bounds plus bounds_status: unknown, in converters/common/metrics.py::metric_bounds_fields.
Chose / instead of: keeping [0, 1], which is what HELM and Inspect did and what makes every record look complete.
Confidence: high — a wrong range is worse than a missing one; the schema makes both fields nullable precisely for this, and the count in source_metadata keeps the gap measurable.
General? yes — this is the rule any converter should follow.
Decision / where: bounds are layered per harness on top of SHARED_METRIC_BOUNDS, in each converter's own table.
Chose / instead of: one global name→range map.
Confidence: high — bleu is the counterexample that forces it: sacrebleu 0–100 in lm-eval, nltk 0–1 for HELM's bleu_1/bleu_4. One map would have to be wrong for one of them.
General? yes.
Decision / where: metric_unit derived from the resolved bounds, with METRIC_UNITS holding only the metrics whose bounds cannot show their scale (bits_per_byte, chrf, ter).
Chose / instead of: listing a unit per metric name.
Confidence: high — a listed unit and a listed range can disagree; a derived one cannot.
General? yes.
Decision / where: a parameter the harness spells into a name takes bounds from the undecorated name but never the id, in metric_config_fields (lookup_name).
Chose / instead of: resolving exact_match@5 to exact-match.
Confidence: high — best-of-five exact match is a different and systematically higher quantity, and the registry gives such metrics their own slug (pass-at-1, recall-at-5). Sharing exact-match would silently average the two in any cross-source query.
General? yes — it is the general rule for @k, _norm and similar decorations.
Decision / where: the registry revision is recorded on resolved ids too, not only unregistered ones (_ID_REVISION).
Chose / instead of: marking only the gaps.
Confidence: high — both are claims about a registry state; a reader of an old record needs to know which state.
General? yes.
Decision / where: id resolution is a hand-verified map + an offline tools/ checker.
Chose / instead of: calling the registry's resolver at conversion time.
Confidence: high — converters must run offline and be reproducible; a network lookup makes last month's conversion unrepeatable.
General? yes.
Decision / where: a HELM run with no usable stats is reported as a per-run failure rather than raising inside the publisher.
Chose / instead of: leaving the exception, which fails the whole --log_path directory.
Confidence: high — SourceConversionResult + save_failure_report + non-zero exit is the repo's documented contract for exactly this.
General? yes.

Coverage: 28 results over the three fixtures — 2 lm_eval, 2 inspect (was 3 — an Inspect std becomes an uncertainty field on the scores it describes rather than a score of its own), 24 helm. Nothing silently dropped; 1 of the 28 comes out without bounds and says so, and 22 of the 45 distinct metric names carry a namespaced rather than canonical id, listed above rather than hidden.

Operator asked about policy calls? Yes — new canonical ids and unbounded metrics, which is why this is needs-a-human. Both listed under "Design agreed in" above: whether namespaced <harness>.<name> ids are acceptable as published data pending registry entries, and whether omitting bounds for a metric no table carries is preferred over the previous [0, 1] default.

A record whose standard error was dropped still validates, so nothing caught the loss. Declare the uncertainty keys per case and require every result to have one.

… known The three harness converters each asserted things about their metrics that the upstream logs do not say. Every HELM and Inspect metric was published as continuous on [0, 1] with higher-is-better, so a perplexity, a standard deviation and an accuracy all came out looking like accuracies. HELM reported `num_samples` from a stat's `count`, which is its train-trial count (1 for nearly every published run, whatever the instance count), and republished the 0.0 spread over that single trial as a measured standard deviation. Inspect took `num_samples` from the length of a sample list that a header-only log does not populate, dropped a standard error of exactly 0.0 as if it were absent, and emitted a scorer's `std` as a score of its own alongside the score it describes. - converters/common/metrics.py holds what a metric's name implies about its range and direction, layered per harness: lm-eval's `bleu` is sacrebleu's 0-100 while HELM's `bleu_1` is nltk's 0-1, so a bare name cannot carry a range. A metric in no table gets no bounds and a `bounds_status: unknown` marker; dispersion metrics get `polarity: not_applicable`, since `lower_is_better` is required and has no "does not apply" value. Each record counts its unknown-bounds metrics in `source_metadata.additional_details`. - HELM counts samples from the per-instance stats, keyed per split and perturbation, falling back to the run's instance count for the worst-case perturbation stats that have none; the trial count moves to `score_details.details.num_train_trials`; the single-trial spread is omitted. A run whose stats are all bookkeeping is now reported per run instead of failing the whole invocation inside the publisher. - Inspect takes `num_samples` from the results header, keeps a standard error of 0.0, and carries a scorer's `std` as `uncertainty.standard_deviation` on the scores it describes rather than as a score, unless it is all the scorer reported.

lm-eval: derive the standard-error method from the aggregation the log records rather than asserting 'bootstrap', and report resamples only for aggregations that are resampled, at the cap lm-eval applies. HELM: keep the across-trial spread out of `standard_deviation`, which the schema defines over per-sample scores, and fall back to HELM's own per-split `num_instances` when a run ships no per-instance stats. Inspect: count the samples each scorer could score, route `bootstrap_stderr` to `standard_error` with its method, and omit an uncertainty that would carry nothing.

`stderr` was covered by the shared bounds and by `DISPERSION_METRICS` while `bootstrap_stderr` was in neither, so a converter publishing the latter as a score claimed an unknown range and an applicable direction. It is a standard error computed by resampling rather than in closed form: non-negative, unbounded above, and no more "better" when low than `stderr` is.

The three harness converters published `metric_name` but no `metric_id`, so nothing tied lm-eval's `exact_match` to HELM's, and a query across sources had to know each harness's spelling. `metric_config_fields` now resolves a metric's name to the eval-card-registry's canonical slug where the registry carries one, and to `<harness>.<name>` where it does not, marked `metric_id_status: unregistered` so it claims no global identity. Both forms record the registry revision they were resolved against. The map is resolved by hand rather than through the registry's resolver: a converter has to run offline and give the same answer on every run. `tools/verify_metric_ids.py` re-does the resolution against a registry checkout and reports a mapped id that has left the registry, a namespaced name that has become resolvable, and a name two entries claim; it also lists the 21 metrics still owed an entry, which is the content of a follow-up registry PR. `metric_kind` and `metric_unit` come along because they answer the questions an id alone leaves open. The unit is derived from the resolved bounds, so lm-eval's sacrebleu `bleu` reports `percent` and HELM's nltk `bleu_1` `proportion` without either being listed. A parameter the harness spells into the name takes its bounds from the undecorated name but never its id: HELM's `exact_match@5` is best-of-five exact match, a different quantity that the registry would give its own slug, and sharing `exact-match` would average the two.

`known_names` was `CANONICAL_METRIC_IDS | METRIC_KINDS`, so a metric with bounds but no `metric_kind` was never checked against the registry at all. `mc2` is one: lm-eval reports it, the converter has its range, and no resolution was ever attempted for its id. It is a probability mass rather than an accuracy, so its absence from `METRIC_KINDS` is correct and cannot be what decides whether it needs an id. The checked set is now the union of every table a converter looks a name up in, which brings the list of metrics owed a registry entry to 22.

…ers do The count in source_metadata.additional_details is what makes an omitted min_score/max_score visible without opening every result. lm_eval and helm report it; inspect did not, and inspect is the one converter whose fixture actually publishes an unbounded metric. SourceMetadata is now built after the results exist, and a case-agnostic test derives the count from the published results so a converter added later inherits the rule.

lm-eval computes brier_score as mean(sum((softmax(lls) - one_hot(gold)) ** 2)), which runs to 2.0 when all the probability mass lands on one wrong class. The shared table declared [0, 1], so the validator would warn that a legitimate score is out of range and the converter suite treats such a warning as a failure.

mrshu

⚒️ review-anvil report

Review decision: COMMENT — This is a careful and well-tested improvement to
metric provenance. The focused suite and all CI matrices are green; four data
semantics edge cases and two small verification-tool refinements remain as
constructive suggestions.
Result: Metric bounds, units, uncertainty, sample counts, and identities now
reflect the upstream harnesses much more faithfully, with unknowns made visible.
Scope: Adds shared metric metadata policy, converter-specific uncertainty and
sample-count handling, canonical/namespaced metric IDs, and an offline registry
verification tool.
Checks: 6 concerns checked; all 6 reproduced, with 2 lowered to low priority
after impact analysis.
Second check: targeted, 2 Codex reviewers; 4 medium findings upheld and 2
tooling findings bounded to low priority.

Earlier review comments

None.

What I noticed

ID	Priority	Topic	Code location	What I noticed
RAV-RUN1-R1-F001	medium	unknown metric polarity	`every_eval_ever/converters/common/metrics.py:204`	Unknown metric names receive `lower_is_better=false`, while only their bounds are marked unknown. A custom loss/error metric can therefore look affirmatively higher-is-better. (inline)
RAV-RUN1-R1-F002	medium	lm-eval bootstrap count	`every_eval_ever/converters/lm_eval/utils.py:77`	The 100-resample cap is selected by the reported metric name rather than its aggregation. An aliased metric using lm-eval's `bleu`, `chrf`, or `ter` aggregation can report the configured count although lm-eval used only 100. (inline)
RAV-RUN1-R1-F003	medium	Inspect metric overrides	`every_eval_ever/converters/inspect/utils.py:349`	Four fields are newly listed as supplement-overridable, but the strict `SupplementalMetricConfig` declares none of them, so every attempted override is rejected before this allowlist runs. (inline)
RAV-RUN1-R1-F004	medium	Inspect uncertainty routing	`every_eval_ever/converters/inspect/adapter.py:234`	If both `stderr` and `bootstrap_stderr` are present, insertion order selects which one survives and the other is removed. The selected metric's parameters are also dropped, including bootstrap counts or clustering information. (inline)

Non-blocking low-priority findings

RAV-RUN1-R1-F005 [low] registry gap report — The verifier removes all
dispersion names from its informational unregistered list, although Inspect
can still emit var and lone dispersion metrics as namespaced results. Its
actual stale/newly-resolvable/ambiguous checks still cover them.
RAV-RUN1-R1-F006 [low] registry provenance — The tool accepts any seed
file but always prints the hard-coded registry revision without checking the
seed checkout's Git revision. Content mismatches are still detected, so this
is primarily a provenance-labeling refinement.

Things to try

[medium] unknown metric polarity — Keep the schema-required boolean, but
add a polarity: unknown marker for names without a verified direction and
distinguish those from known higher-is-better metrics. Harness-provided
direction can then clear the marker. (RAV-RUN1-R1-P001; covers
RAV-RUN1-R1-F001)
[medium] lm-eval bootstrap count — Apply the cap using aggregation, as
standard_error_method() already does, and add an aliased-name/bleu
aggregation regression test. (RAV-RUN1-R1-P002; covers
RAV-RUN1-R1-F002)
[medium] Inspect metric overrides — Add the four fields to
SupplementalMetricConfig and test a real override. For metric_id, updating
the generated registration status/revision atomically would keep identity
metadata consistent. (RAV-RUN1-R1-P003; covers RAV-RUN1-R1-F003)
[medium] Inspect uncertainty routing — Handle multiple standard-error
metrics deterministically (or preserve the alternate in details), and carry
the chosen metric's bootstrap count/clustering parameters into uncertainty
metadata. (RAV-RUN1-R1-P004; covers RAV-RUN1-R1-F004)
[low] registry gap report — Exclude only dispersion metrics guaranteed
not to be emitted, or list fallback-emitted dispersion IDs separately.
(RAV-RUN1-R1-P005; covers RAV-RUN1-R1-F005)
[low] registry provenance — Compare the seed checkout's revision with the
pinned constant, or label the printed revision explicitly as the map's
expected provenance. (RAV-RUN1-R1-P006; covers RAV-RUN1-R1-F006)

Run details

Target: PR #246 at f10d4d7aab6df0454a9f957dd82c58b438976f5e
(16 files, +1865/-129)
Run ordinal: 1
Rounds: 1/1 completed; adaptive off; material findings remain
Mix: 3 Codex reviewers
Focus: bounds, polarity, uncertainty, sample counts, metric identity,
registry verification, and positive suggestion-oriented language
Earlier review comments: none
Finding counts: 0 critical, 0 high, 4 medium, 2 low, 0 nit
Verification: exact-head focused converter run — 55 passed, 24
optional-extra skips; GitHub Actions passed all four core/full locked/loose
matrices
Reproduction: one batched Codex verifier confirmed 3 single-reviewer medium
candidates and lowered 2 verifier-tool candidates to low; consensus covered
the remaining medium candidate
Second check: targeted; reviewers=2; kept=4/lowered=2/removed=0; approval
changed no

Reviewed with review-anvil.

mrshu · 2026-08-11T00:57:37Z

+
+    if bounds is None:
+        return {
+            'lower_is_better': name in LOWER_IS_BETTER,


RAV-RUN1-R1-F001 [medium] — suggestion: Could unknown direction be marked explicitly here, similarly to dispersion's not_applicable marker? For any unrecognized loss/error metric this required false reads as an affirmative higher-is-better claim, while the current additional details only qualify the bounds. Keeping the boolean but adding polarity: unknown until a harness or supplement supplies direction would make the fallback safer for ranking consumers.

mrshu · 2026-08-11T00:57:37Z

+    """How many resamples went into a bootstrapped standard error."""
+    if configured_iters is None or aggregation not in BOOTSTRAP_AGGREGATIONS:
+        return None
+    if metric_name in CAPPED_BOOTSTRAP_METRICS:


RAV-RUN1-R1-F002 [medium] — suggestion: lm-eval applies this 100-resample cap to the aggregation callable, so an aliased metric can use aggregation: bleu without being named bleu. Keying this check on aggregation (as standard_error_method() already does) would keep num_bootstrap_samples faithful for custom metric names too.

mrshu · 2026-08-11T00:57:37Z

    "has_unknown_level",
    "min_score",
    "max_score",
+    "metric_id",


RAV-RUN1-R1-F003 [medium] — suggestion: These four new fields look intentionally overrideable, but the strict SupplementalMetricConfig does not declare them, so exact probes reject each as extra_forbidden before this allowlist runs. Adding matching model fields and an end-to-end override test would make this useful correction path reachable; a metric_id override can also refresh its generated status/revision atomically.

mrshu · 2026-08-11T00:57:38Z

                )

-            stderr_value = next(
+            stderr_value, stderr_method = next(


RAV-RUN1-R1-F004 [medium] — suggestion: If a scorer carries both stderr and bootstrap_stderr, this next(...) makes the published value and method depend on mapping order while both source entries are later removed. Could the collision have deterministic policy (or preserve the alternate in details), and could the chosen metric's params populate bootstrap count/clustering metadata? That would retain the useful uncertainty provenance this PR is adding.

borgr added 5 commits August 10, 2026 12:27

Require every converted score to carry its uncertainty

a142d90

A record whose standard error was dropped still validates, so nothing caught the loss. Declare the uncertainty keys per case and require every result to have one.

This was referenced Aug 10, 2026

helm: drop the codec import and relax the nltk pin, fixing the extra on Python 3.14 #243

Open

Test that a converted record actually passes the merge gate #244

Open

borgr added 3 commits August 10, 2026 12:52

mrshu reviewed Aug 11, 2026

View reviewed changes

Navigation Menu

Uh oh!

Report metric bounds, uncertainty and ids the way each harness computed them - #246

Report metric bounds, uncertainty and ids the way each harness computed them#246
borgr wants to merge 8 commits into
fix/converter-metric-naming-3fieldsfrom
fix/converter-metric-config

borgr commented Aug 10, 2026 •

edited

Loading

Uh oh!

mrshu left a comment

Uh oh!

mrshu Aug 11, 2026

Uh oh!

mrshu Aug 11, 2026

Uh oh!

mrshu Aug 11, 2026

Uh oh!

mrshu Aug 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

borgr commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What / source

Why the map is hand-resolved, and how you check it

Checklist

Review lane

Decisions & coverage

Uh oh!

mrshu left a comment

Choose a reason for hiding this comment

⚒️ review-anvil report

Earlier review comments

What I noticed

Things to try

Uh oh!

mrshu Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

mrshu Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

mrshu Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

mrshu Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

borgr commented Aug 10, 2026 •

edited

Loading