fix(cli): the metadata rubric records a linter crash instead of scoring it 100 / A

[os-litant]

Fixes #15658

The card's open question, settled by measurement first

The card refused to pre-answer its own question and was right to: is the throw reachable at all, on a value that already survived normalizeStackInput and ObjectStackDefinitionSchema.safeParse? If not, the honest repair was to delete the catch, not widen the report.

World (i). The throw is REACHABLE, and not exotically. A driven input:

scoreMetadata({ apps: [{ name: 'todo_app', label: { en: 'Todos', 'zh-CN': '待办' } }] })

safeParse returns success: true — apps[].label is I18nLabelSchema, a union of z.string() and an inline locale map, and the map form is the platform's own way to write a localized label. lintConfig then throws TypeError: Cannot read properties of undefined (reading 'toUpperCase'), because checkLabelCase indexes the label as a string. On origin/main the scorer swallowed that into issues: [] and returned:

{"score":100,"grade":"A","valid":true,"counts":{"schemaErrors":0,"errors":0,"warnings":0,"suggestions":0}}

views[].list.label with the same locale map does it too.

How the input was found, with its control. A single-value mutation sweep over examples/app-todo's normalized config — every path, eight hostile values, 15,728 mutations — kept 3,064 schema-valid, of which exactly two crash lintConfig (the two above). Positive control in the same run: 241 schema-INVALID mutations also crash it, so the detector fires and the two hits are not an artefact of an empty scan. A second sweep over the four bundled eval-corpus fixtures (2,745 mutations, 241 schema-valid survivors, control 314) found none — those fixtures declare no apps and no views.

So the repair is the one this repo has chosen twice before (#10653, #10123): make the failure visible in the output, keep the guard.

⚠️ The crash itself is a separate defect and is deliberately NOT repaired here. checkLabelCase assuming a string is a bug in packages/cli/src/commands/lint.ts; repairing it in this PR would delete the very input that proves this path reachable, and it is a different defect class. Filed with its measurement as #15880.

Both published faces, measured — and one correction to the card

The card lists two faces. Driven end to end against a CLI built from this tree, they behave differently, and the difference is worth recording:

os lint --score — already loud, before and after. The command calls lintConfig(normalized, { sduiManifest }) at its own step, before scoreMetadata, so the same throw kills the run first:

$ os lint --score            → ✗ Cannot read properties of undefined (reading 'toUpperCase')   EXIT=1
$ os lint --score --json     → {"error":"Cannot read properties of undefined (reading 'toUpperCase')","conversions":[]}   EXIT=1

The silent 100 the card predicts for this face is not reachable through this input: the only difference between the two calls is sduiManifest, and the command's call is the superset. That is a correction to the card's blast-radius bullet 1, not to its verdict.

os lint --eval — the face where a failure became a PASS. It reaches scoreMetadata directly, with no earlier lint call. Same generator, same tree, before and after (before = this branch with the catch body ablated back to issues = [], rebuilt, so the dist/ the CLI runs really carried the old behaviour):

	ok	passed	failed	meanScore	case verdict	exit
before	true	5	0	100	100 / A / valid: true, counts all 0	0
after	false	0	5	0	0 / F / valid: false, errors: 1, lintError set	1

A generator whose every output crashed the linter earned a green eval and exit 0. That is the pass-flip the card graded correctly.

What changed

packages/cli/src/lint/score.ts only. The catch no longer discards; it records, in every carrier a consumer might read, because reading any one of them has to be enough:

• lintError?: string — new optional field on MetadataScore, the thrown message. Set only when the linter could not run; absent when it ran and reported errors, which is a lint verdict rather than a missing one. Optional on purpose: unscorableScore() in metadata-eval.ts constructs a MetadataScore literal, and a required field would have forced an edit into a file another card is fenced to.
• A synthetic error issue, rule: 'rubric/lint-crashed' (exported as LINT_CRASHED_RULE) — so issues, counts.errors and valid carry it. This is what makes the eval harness fail the case with no edit to metadata-eval.ts: its passed reads counts.errors, and would never have seen a new field. It is also what still fails the case at --eval-min 0, where the score alone stops discriminating — pinned by its own test.
• score: 0 / grade F — the only channel os lint --score --json publishes ({ score, grade }), and the same refusal unscorableScore() already gives an eval case there was nothing to judge, for the same stated reason: a clean number nothing earned is the worst possible output.

The schema half is untouched and still reported: schemaErrors and counts.schemaErrors say exactly what the parse found. That was the defensible half of the original comment's intent, and it survives.

Why this shape (the four axes)

• ② 长远合理性 (weighted highest). The repo has already ruled on this exact question one module over: unscorableScore() returns 0 / F / valid: false and its docblock says why — ⛔ deliberately not scoreMetadata({}), because "a clean number nothing earned" is the failure mode. Reusing that verdict for "the rubric did not run" keeps one answer in the codebase instead of two. Adding a lower number without a flag would have been the alternative the card warns about (one wrong number for another); adding a flag without the number would have left os lint --score --json, which publishes nothing but score and grade, exactly as misleading as before.
• ③ 防 AI 写代码犯错. A verdict that cannot distinguish "checked and clean" from "never checked" is precisely the input that teaches an agent the wrong thing — and the eval harness is an agent-facing gate. Three carriers means no single-field read can recover the comfortable answer.
• ① 实际业务需求, measured. Not hypothetical: apps[].label accepting a locale map is a shipped authoring feature, and the crash it causes is reachable from a schema-valid stack today.
• ④ 不扩散需求. One file changed, one optional field, one rule id, no new option, no new command, no edit to the sibling card's file.

Ablation

Prediction written before the run (catch body reverted to issues = [], marker injected, nothing else moved): 6 RED by name, 3 GREEN in the same file plus all of score.test.ts and metadata-eval.test.ts, and the two schema assertions inside keeps the schema verdict staying green because the parse half is untouched.

Mutation proven on disk before the run — removed text rule: LINT_CRASHED_RULE 1 → 0, injected marker ABLATION-15658 0 → 1, blob hash eef1cc1b → 4c74b941. Restore under an EXIT INT TERM trap, proven after: disk hash eef1cc1b equals the HEAD blob, and git diff HEAD is empty. The test reaches the subject through source (test/score-lint-crash.test.ts imports ../src/lint/score.js), and packages/cli/dist held 0 entries at mutation time; for the CLI-face table above, where dist IS the path, the ablated build was verified to carry the marker (1) and the restored build to have lost it (0).

Result — exactly the prediction:

Test Files  1 failed | 2 passed (3)
     Tests  6 failed | 25 passed (31)

with the six failures being the six named ones, and keeps the schema verdict failing only on expected undefined to be 'boom' at its third assertion.

Verification — exit codes captured before any pipe

At c4f2e7ae671, working tree clean.

• pnpm --filter '@objectstack/cli^...' build — VERDICT command-exit 0 (the dependency closure; @objectstack/spec and @objectstack/lint resolve through exports to dist/).
• pnpm --filter @objectstack/cli exec vitest run --maxWorkers=2 over score-lint-crash, score, metadata-eval, lint-eval-json-unscorable-stack.e2e, lint-view-label, lint-namespace-prefix — 6 files / 63 tests passed, VERDICT command-exit 0.
• pnpm --filter @objectstack/cli typecheck — VERDICT command-exit 0; check:test-typecheck: OK. Both changed files are really in that program: tsc -p packages/cli/tsconfig.test.json --listFiles names score-lint-crash.test.ts (1) and src/lint/score.ts (1), so this is a measurement and not a NOT MEASURED.
• pnpm lint — the repo-wide eslint . --no-inline-config, not a narrowed run: VERDICT command-exit 0, 77s.
• 18 gates from scripts/pm/dispatch-gates.mjs (derived from the delivered diff at this commit, --repo objectstack-ai/objectstack asserted), each captured as cmd > log 2>&1; EXIT=$?: nul-bytes, verify-stand-in, single-claim-paths, cli-test-child-env, cross-package-test-inputs, test-source-alias, type-source-resolution, changeset-gate-self-tests, check-changeset-no-major, check-empty-changeset, check-adr-0087-registration, check-comment-mask-adoption, objectql-double-limit, where-matcher, error-code-casing, doc-authoring, objectui-changeset, check-keyed-text-bounds — all EXIT=0, quoting their own verdict lines, e.g. check-nul-bytes: OK (scanned 7655 text file(s) …), ✓ This diff introduces no 'major' bump., check-test-source-alias OK — 72 packages with tests scanned. The remaining families the derivation lists are packages/**-glob matches CI runs in full; that half is CI's, by the dispatch's own rule.

Clause ② (契约复审) — declared from the delivered diff

Mechanical / path limb — YES. lintError is a new key on a published payload: os lint --eval --json emits the whole report, and results[].score is the MetadataScore object. The key is in the JSON above, measured, not inferred. The packages/spec/src/** path leg does not apply — no spec file is touched.

Non-mechanizable conformance limb — YES, and driven. The before/after table is exactly the shape the doctrine names: an input class that passed on a shipped face (ok: true, exit 0) now fails (ok: false, exit 1). Not a re-reading of a borderline case — a measured verdict flip.

Bump: minor, per the Check Changeset step's WHICH LEVEL section (scripts/check-changeset-no-major.mjs:54) — a new accepted key on a published surface takes at least minor, and the commit type may raise but never lower it. Not major: the gate refuses it, and check-adr-0087-registration confirms this diff declares no breaking changeset.

Scope

#15578 is a separate card and its repair has landed on main already; nothing here touches metadata-eval.ts. The label-case crash that makes this path reachable is filed as #15880 and is not repaired here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

4 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see

• the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 22 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 8c7b2394f27095995bc0d5b487cc0242319511c2 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from f2e617487b26223b26a774c180680fb5864adf22 — the merge of head c4f2e7ae671b3e9044236cc4e79cc56f6eafbadd into base 8c7b2394f27095995bc0d5b487cc0242319511c2, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin f2e617487b26223b26a774c180680fb5864adf22 && git checkout f2e617487b26223b26a774c180680fb5864adf22
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 8c7b2394f27095995bc0d5b487cc0242319511c2 c4f2e7ae671b3e9044236cc4e79cc56f6eafbadd && git checkout -B drift-repro 8c7b2394f27095995bc0d5b487cc0242319511c2 && git merge --no-ff c4f2e7ae671b3e9044236cc4e79cc56f6eafbadd

node scripts/docs-audit/affected-docs.mjs --json 8c7b2394f27095995bc0d5b487cc0242319511c2

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

[os-litant]

Contract review (clause ②) — PASS

Reviewed at head c4f2e7ae671b3e9044236cc4e79cc56f6eafbadd (merge-base with origin/main 59953d5a3). Posted as a COMMENT, not an approving review: GitHub refuses APPROVE on a PR authored under this same account, and agent seats do not submit approving reviews in any case. This is the in-seat contract review the card was dispatched under; the label stroke and landing are the seat's own and are not taken here.

Independence pair (C4):

• Implemented-by: branch claude/issue-15658-score-lint-crash-visible (subagent dev)
• Reviewed-by: session_01D47qPfEWVPmhguWgBZCi5N

Governed surfaces touched: none (.changeset/…, packages/cli/src/lint/score.ts, packages/cli/test/score-lint-crash.test.ts). metadata-eval.ts untouched, content/docs/releases/ untouched, no packages/spec/src/** file touched.

Clause ② — re-judged from the delivered diff, per limb

Mechanical limb — YES, and narrower than declared. packages/cli/src/index.ts re-exports commands only; score.ts is not on the package's . export and the exports map has no deep-import subpath ⇒ the mechanical limb's "new exported symbol on an index" leg does not apply, and the mechanical YES rests solely on the os lint --eval --json wire key. That key is real: driven through a dist built from this head, results[].score.lintError is an own property on every crashed case (5 of 5 lines on the crash run, 0 on the clean control), and the new rule value rubric/lint-crashed rides the same payload in results[].score.issues[]. LINT_CRASHED_RULE and MetadataScore are not npm-public TypeScript API. The packages/spec/src/** path leg is correctly declared not applicable.

Conformance limb — YES, re-derived. The input class is "a schema-valid stack on which lintConfig throws"; the concrete members are apps[].label and views[].list.label carrying an inline locale map. Same process, same input, OLD (merge-base) scorer vs NEW scorer:

	score	grade	valid	counts.errors	lintError	issues[].rule
OLD, apps[].label = {en, zh-CN}	100	A	true	0	—	[]
NEW, same input	0	F	false	1	set	rubric/lint-crashed
NEW, views[].list.label = {…}	0	F	false	1	set	rubric/lint-crashed
OLD and NEW, string label (control)	100	A	true	0	—	[]

The unchanged harness (metadata-eval.ts, passed reads counts.errors) fails the case at minScore 75 and at 0 (passed:false, ok:false, meanScore:0); the clean control passes at both. On the wire (os lint --eval --json --generator): crash generator EXIT=1, ok:false, 0/5 passed, meanScore 0; clean-generator control EXIT=0, ok:true, 5/5, meanScore 100. The wire-level before (exit 0) is derived, not driven by me: OLD scorer returns 100/A/valid:true with every count zero, passed (unchanged) is therefore true, ok true, and lint.ts exits 1 only on !report.ok — the dev's ablated-build table is taken on trust for that one row.

minor vs major — explicit verdict: minor is correct; not major; and no BREAKING banner or ADR-0087 disposition is owed

I did not take this from the green gate. The changeset gate checks the declaration's shape; the ADR-0087 gate is, by its own header, blind to omission — so "the gate confirms this declares no breaking change" is a tautology, not evidence. The ruling below stands on the merits.

1. The level axis. Under the maintainer's 2026-09-04 WHICH LEVEL ruling (pr-automation.yml, Check Changeset step) and the header of check-changeset-no-major.mjs, the bump level does not carry breaking-ness during the launch window: major is refused, and breaking-ness is carried by the BREAKING banner plus the ADR-0087 disposition. A new accepted key and a new accepted value on a published payload set the floor at minor, and a fix( type cannot lower it. minor is exactly what the ruling prescribes; major would be wrong on this axis regardless of the next question.
2. The breaking axis — is a BREAKING banner owed? No.
   • The repo's own definition (AGENTS.md rule 3): a breaking changeset "removes or renames anything an author can write (a spec key, an export, a config field)". This PR removes and renames nothing. Every existing key keeps its shape and meaning; the additions are optional and additive; a consumer that parsed the old payload parses the new one.
   • The behaviour flip is confined to an input class whose prior output was a defect against the rubric's documented contract — score.ts says a stack is good "exactly when it (a) parses and (b) is clean under the data-model lint rules"; a stack whose lint never ran was never established as (b). The 100 / A / exit 0 was unearned, not promised. Correcting an incorrect verdict for one input class is a fix, and this repo has twice shipped the same-family correction that way: #15576 and #15659 both flipped a published number from 100 to 0 on an input class and landed as patch with no banner.
   • The same input already failed the tool's primary face before this PR: os lint and os lint --score exit 1 on it, because the command-level lintConfig call (lint.ts:583) runs before scoreMetadata (:613). No consumer could hold a consistent green on this stack across the tool; only the eval face disagreed with the rest of it. Aligning it narrows no contract.
   • There is nothing to migrate: an ADR-0087 disposition presupposes an author-side rewrite, and an author who meets rubric/lint-crashed has hit a linter bug (#15880), not a metadata shape they must change.
   • What I do acknowledge: a CI that ran os lint --eval --generator against a generator emitting localized app or view labels goes from green to red. That is the correct verdict for that generator under the rubric and is the pass-flip #15658 was filed to close. It deserves plain words in the changeset body — which it has ("a stack half of whose rubric never ran came back as 100 / A") — not a banner whose meaning here is "you must rewrite something".

I18nLabelSchema — re-derived, admits the locale map

I18nLabelSchema = z.union([z.string(), InlineLocaleMapSchema]) (packages/spec/src/ui/i18n.zod.ts:247); InlineLocaleMapSchema = z.record(<key matching INLINE_LOCALE_KEY>, z.string()), and INLINE_LOCALE_KEY = /^(?!(?:key|defaultValue)$)(default|[A-Za-z]{2,3}(-[A-Za-z0-9]{2,8})*)$/ admits en and zh-CN. AppSchema.label: I18nLabelSchema (app.zod.ts:1291); ObjectStackDefinitionSchema.apps: z.array(AppSchema).optional() (stack.zod.ts:333). Executed at this head: safeParse success on apps[].label = { en, 'zh-CN' } and on views[].list.label = {…}; control on the schema axis: objects[].label is z.string() and the same parser rejects the map (objects.0.label: Invalid input: expected string, received object) — so the pass is not a parser that admits everything. Then lintConfig throws TypeError: Cannot read properties of undefined (reading 'toUpperCase') on both — checkLabelCase (commands/lint.ts:101-102) indexes label[0] on an object; control with a string label returns 0 issues and does not throw. Reachability (world i) is established; the "delete the catch" branch of the card is closed.

On the sweep's control. The 241 schema-INVALID crashers discriminate on the detector axis — they show the harness observes a throw from lintConfig at all, so "2 hits in 3,064" is not an always-zero detector. They do not discriminate on the filter axis (that "schema-valid" survivors really passed safeParse); that axis is covered by the objects[].label rejection, which I reproduced. The sweep's counts themselves are NOT MEASURED here (the sweep script is not in the PR); the card's question does not depend on them — one driven input settles reachability, and I reproduced both hits.

The three carriers — three readers, not three spellings

• Synthetic error issue — the only carrier the unchanged harness reads (passed ⇐ counts.errors === 0), the only one valid reads, and what the human --eval line prints (e.g. rule: message). Measured: at --eval-min 0 the case still fails only because of it.
• score: 0 / F — the only carrier meanScore reads (it sums score.score alone) and what the human per-case line prints; without it a crashed case contributes 100 to the mean, which re-opens #15659's exact defect. It is also the only content os lint --score --json publishes ({score, grade}, lint.ts:627) — on the current input class that face never reaches the scorer, but the scorer's contract is for any throw from any rule.
• lintError — the typed cause, and the only carrier that distinguishes "the linter reported an error" from "the linter never ran" without matching a rule-id string.

Sound. One acknowledged cost: counts.errors now includes a non-lint event; acceptable, since the rule id and lintError disambiguate and a separate count would widen the counts shape.

lintError optional — right design, wrong stated reason. Right on its own merits: (i) generationError?: string on the same report's case result is already the present-when-set shape, so lintError? matches the idiom beside it; (ii) a required field would force unscorableScore() to pick a value for a case whose linter never ran for an upstream reason — undefined there would assert "ran clean", which is false; absent is the honest value; (iii) the docblock's "absent means it ran" is the semantics. The PR's stated reason — that a required field would have forced an edit into a file another card is fenced to — is not a reason a published type should take a shape; had required been right, the fence would have had to yield. It did not have to, because optional is right anyway. Recorded; no change requested.

Scope fence — leaving the crash to #15880 is right

On stronger grounds than the one stated. The scorer's contract is for any throw from any rule, and the test pins it through a mocked linter, so this PR's guarantee does not depend on #15880's fate; the crash's repair is a product/contract choice (guard on typeof label === 'string', or resolve the default-locale entry — and the objects[].label vs apps[].label asymmetry) that the issue correctly routes to triage; and no user is left worse off — os lint already exited 1 on the input before this PR, and the eval face now agrees with it. "Repairing it would delete the input that proves reachability" is the weakest of the three reasons (the suite does not rely on that input). Not a blocker.

Measurements (exit codes captured before any pipe; worktree at this head, git status clean)

leg	result
vitest run — score-lint-crash, score, metadata-eval	EXIT=0, 3 files / 31 tests passed
same-process OLD-vs-NEW probe (source via tsx; OLD = merge-base blob of score.ts)	table above; controls green on both axes
dist carries the change	rubric/lint-crashed ×1 in dist/lint/score.js, ×0 in the merge-base blob; on-disk score.ts blob = HEAD blob eef1cc1b
os lint --eval --json --generator via bin/run.js (dist)	crash EXIT=1 / clean control EXIT=0; details above
check-changeset-no-major / check-adr-0087-registration / check-empty-changeset (--base origin/main)	EXIT=0 / 0 / 0 (re-run here, not read from CI)
CI at head	Test Core (6/6), TypeScript Type Check, Build Core, Check Changeset, Dogfood gates: all success

Resolution path, stated: the unit probe and the tests reach score.ts through source; the wire runs reach it through a dist built from this head (JS-only, OS_SKIP_DTS=1, lint-command closure of 16 packages plus @objectstack/cli). That partial build leaves oclif ModuleLoadError warnings on stderr for unrelated commands (cloud:*, data:*, …); none for lint, and stdout is well-formed JSON.

NOT MEASURED (neither pass nor red): the mutation sweep's counts; the dev's ablation (6 RED / 25 GREEN prediction) — not re-run; the wire-level before row (derived from the OLD scorer's output and the unchanged passed/exit logic, not driven); pnpm lint and pnpm typecheck locally — not re-run (CI's TypeScript Type Check is green at this head; the local cli build ran --noCheck); DTS emission — skipped in the local build.

Non-blocking notes

• The PR body cites the ADR-0087 gate as confirming non-breaking-ness; that gate cannot confirm it (blind to omission by design). The reasoning above is what carries the ruling; the outcome is unchanged.
• An Error('') thrown by a rule yields lintError: "", which a truthiness-reading consumer would miss; the other two carriers still hold. Exotic; not worth a change.

---

Generated by Claude Code