Skip to content

skills(platform): delete the placeholder eval README and drop its ceiling row (#14296 item 2) - #14681

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-14299-platform-eval-stub-removal
Sep 2, 2026
Merged

skills(platform): delete the placeholder eval README and drop its ceiling row (#14296 item 2)#14681
os-zhuang merged 2 commits into
mainfrom
claude/issue-14299-platform-eval-stub-removal

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Fixes #14299

Ruling (not re-adjudicated here). Item 2 of the eval-stub decision card #14296 was ruled A: planned-eval stubs are deleted, their ceilings step down, real fixtures stay, and no new eval packages are written. This is round 2 of #14299 — finding PLAT-H-01, the item round 1 (PR #14351, landed as Part of #14299) deferred. It is this card's last in-scope item: the other deferred finding, PLAT-G-02 (the references/ placement question), is carried by the catalog-wide card #14655, which is why the first line here is a closing keyword and round 1's was not.

Same two-file shape as the merged precedents #14585 (query) and #14578 (automation), and as #14652 (ai).

Pre-deletion verification

The ruling keeps real fixtures, so the directory had to be proven fixture-free before anything was touched. Three readings, all on the branch base 20b88391:

  1. find skills/objectstack-platform/evals returned the directory and README.md — nothing else, zero subdirectories.
  2. The file is the placeholder, not a fixture. Its own words: :3 "This directory is reserved for future skill evaluation tests"; :11 "When implemented, evals will follow this structure"; :41 "⚠️ Not yet implemented — This is a placeholder for future development". The tree it prints at :13-29 names ten eval files; each was searched for by name across the whole tree and all ten return 0 matchestest-definestack-keys, test-manifest-required-fields, test-template-selection, test-driver-selection, test-hono-integration, test-lifecycle-phases, test-service-registry, test-kernel-events, test-cli-commands, test-litekernel-testing.
  3. Nothing reads the path. git grep -n "objectstack-platform/evals" returns exactly one hit repo-wide — scripts/check-skills-token-ratchet.mjs:420, the ceiling row itself. The package links evals from nowhere: git grep -n evals -- skills/objectstack-platform/ returns only lines inside the deleted file.

No real fixture was found, so the ruling's precondition holds and the stub is deletable.

The diff — two files, 50 deletions, nothing else

File Change Lines Tokens
skills/objectstack-platform/evals/README.md deleted (the whole placeholder) −49 514 → 0
scripts/check-skills-token-ratchet.mjs its one ceiling row dropped (line 420: ['skills/objectstack-platform/evals/README.md', 514],) −1 ceiling 514 removed

Proved mechanically rather than asserted: the CEILINGS map parsed out of the base blob and out of the head file gives 31 rows / 165,532 ceiling tokens before → 30 rows / 165,018 after; removed keys = exactly skills/objectstack-platform/evals/README.md (514); added keys = none; rows whose value changed = none. No other row moved, no ceiling was raised, no row was added, and no prose was edited anywhere.

The // objectstack-platform section comment above the row is left standing and untouched, per the flight constraint. It is not orphaned by this change: three rows remain under it (references/plugin-hooks.md, rules/plugin-lifecycle.md, rules/service-registry.md).

Token readings for the governed skills/** surface

Reading Before After Delta
The changed skills file (evals/README.md) 514 0 (deleted) −514
Package skills/objectstack-platform 20,673 20,159 −514 (−2.5%)
Package ceiling sum 20,865 20,351 −514
Published bundle (whole shipped tree) 157,662 157,148 −514

Pure deletion — nothing was added, so there is no expansion to budget against. 514 = ceil(2055 utf8 bytes / 4), the ratchet's own price convention, and the file sat at its ceiling with headroom 0.

Reverse verification — red, then green

Both readings taken with the exit code captured by redirect-then-capture, never after a pipe.

Leg 1 — file deleted, ceiling row still present ⇒ the gate must go RED, not skip. node scripts/check-skills-token-ratchet.mjs, EXIT=1, and it printed its documented missing-input line verbatim:

✗ check-skills-token-ratchet: cannot read skills/objectstack-platform/evals/README.md — red, not a skip (#4690).
✗ check-skills-token-ratchet: 1 of 42 published bundle file(s) failed their check.

Leg 2 — row dropped ⇒ green. Same command, EXIT=0:

✓ check-skills-token-ratchet: 30 authored bundle file(s) within their ceilings; 11 generator-owned file(s) measured, not ratcheted.

Leg 3 — the script's own test suite. node scripts/check-skills-token-ratchet.mjs --self-test, EXIT=0:

✓ check-skills-token-ratchet self-test: 64 cases pass.

The gate script is edited by this diff, so its own suite is owed on top of the derived family. --self-test is that suite: no vitest file names this script (git grep -l check-skills-token-ratchet over test globs returns nothing), and the three other scripts that do read it — check:ratchet-remedy-authority, check:doc-authoring, scripts/pm/bare-root-worklist.mjs --self-test — are all in the derived family below and all exit 0.

Gates

Head sha a8ceb57b — the merge commit, and the head every reading below was taken on. origin/main moved from the branch base 20b88391 to 2aa8456c while this was in flight, so it was merged in, not rebased; the merge was conflict-free, and neither the hot file nor any file of this package had moved on main. The PR's own delta is unchanged by the merge: 2 files, 50 deletions.

Family derived from the real change set at the merge head, not hand-written: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands24 families (22 by path + 2 by change kind — this diff edits a gate script). Byte-identical to the list derived before the merge. All 24 run through scripts/pm/os-verify-lock.sh, slot issue-14299-r2; every exit code captured before any pipe.

22 of 24 exit 0, plus check:skill-docs and check:nul-bytes run beyond the family. 2 of 24 are NOT MEASURED (exit 3), both by their own prerequisite text.

Gate Exit Its own verdict line
node scripts/check-skills-token-ratchet.mjs 0 ✓ 30 authored bundle file(s) within their ceilings; 11 generator-owned file(s) measured, not ratcheted.
pnpm check:ratchet-remedy-authority 0 OK 184 scripts swept; 12 mark the expanding remedy ⛔ MAINTAINER-ONLY, 6 turn it down outright, 166 hand out no ratchet-expanding remedy.
node scripts/pm/bare-root-worklist.mjs --self-test 0 OK 57 live row(s), 49 unreachable as spelled, 49 recorded verdict(s) — none stale, none missing, none contradicted.
pnpm check:pm-dispatch-gates 0 ✓ dispatch-gates self-test: 1240 cases pass.
pnpm check:skill-identifier-liveness 0 OK — Leg 1: 465 citation(s) over 40 published file(s) ... Leg 2: 8 registered exhaustive section(s), 0 ledgered gap(s).
pnpm check:skill-compatibility 0 ✓ 11 SKILL.md file(s) reconciled against 79 workspace packages
pnpm --filter @objectstack/spec check:skill-docs 0 ✅ Skill docs in sync — no regeneration demanded; this diff touches no frontmatter
pnpm check:skill-frame-sync 0 binding sentence present in all 4; 4 count mention(s) agree; 34 markdown files scanned
pnpm check:corpus-claim-drift 0 self-test plus live sweep, both clean
pnpm check:doc-authoring 0 ✓ 14500 customer-facing string(s) across 710 spec sources clean
pnpm check:pm-governed-merges 0 ✓ --self-test: 243 assertions + live: the real generator declared 9 output(s) and certified this tree
pnpm check:watch-hint-literal 0 ✓ 45 declaration(s) across 4 rostered name(s) ... no unrostered spelling of the idiom in the tree.
pnpm check:role-word 0 ledger unchanged — 42 baselined file(s), 121 occurrence(s)
pnpm check:agent-test-spelling 0
pnpm check:bash32-floor 0
pnpm check:cli-command-ids 0
pnpm check:cross-package-test-inputs 0
node scripts/check-cross-package-test-inputs.mjs 0
node scripts/check-ci-filter-parity.mjs 0
node scripts/check-shard-attestation.mjs 0
pnpm check:entry-guard 0
pnpm check:parse-guard 0
pnpm check:pnpm-filter-targets 0
pnpm check:nul-bytes (beyond the family) 0 OK (scanned 7994 text file(s) ... no raw ASCII control bytes), plus a direct control-byte scan of the edited script: no hits

NOT MEASURED (exit 3 — a prerequisite this flight does not supply, never a red)

Gate Prerequisite it names Who supplies it
node scripts/check-test-completeness.mjs PREREQUISITE NOT MET — this gate grades a saved 'turbo run test' log, and no log was named. CI tees the log and passes the path
pnpm --filter @objectstack/lint run check:doc-formula-expressions PREREQUISITE NOT MET — the workspace package @objectstack/formula is not built the CI build job

Neither is touched by this diff — it adds no test and no formula expression — and both print, in their own words, that the local result "says NOTHING about what it gates ... It is NOT a finding". Recorded as NOT MEASURED, never counted as green and never called a flake.

Housekeeping

  • No changeset, and skip-changeset is applied — checked against scripts/check-empty-changeset.mjs's own enumeration rather than by habit. That enumeration reads "It releases nothing (.github/, .claude/, skills/, docs/, content/, examples/, tests-only, and the like) -> delete the changeset and apply the 'skip-changeset' label (route 2)": skills/ is named verbatim, and scripts/check-skills-token-ratchet.mjs is a CI-internal gate script that publishes from no package, which is the "and the like" case. The same file pins a skills/**-only PR carrying no changeset at all as a green fixture (its GREEN 3 self-test case).
  • No needs:contract-review: a deletion of a placeholder that documents nothing about runtime behaviour makes no operator or contract-semantics claim.
  • Governed face (skills/**) ⇒ this PR stays DRAFT. Review requests are the seat's step; it is not flipped ready and not enqueued.
  • content/docs/releases/ untouched. Assignee and card labels untouched.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1


Generated by Claude Code

…ling row

The file was a stub for evals that were never written: it says the directory
is "reserved for future skill evaluation tests", that "When implemented, evals
will follow this structure", and that it is "Not yet implemented — This is a
placeholder for future development". The tree it prints names ten eval files,
none of which exists anywhere in the repository.

Nothing read the path. Repo-wide, the single reference to
skills/objectstack-platform/evals was its own ceiling row in the token ratchet,
so the row goes with the file and the package ceiling steps down by 514 tokens.
No other row is touched, no ceiling is raised, and no prose changes.

Verified before the deletion (the ruling keeps real fixtures, so the directory
had to be proven fixture-free): the evals directory held README.md and nothing
else, with zero subdirectories.

Reverse verification: with the file deleted and the row still present,
check-skills-token-ratchet exits 1 with "cannot read
skills/objectstack-platform/evals/README.md — red, not a skip (#4690)"; with
the row dropped it exits 0 over 30 authored bundle files; --self-test passes
64 cases.

Ruled A for item 2 of the eval-stub decision card (#14296): planned-eval stubs
are deleted, their ceilings step down, real fixtures stay, no new eval packages.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
@os-zhuang
os-zhuang marked this pull request as ready for review September 2, 2026 22:52
@os-zhuang
os-zhuang added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit ba1806d Sep 2, 2026
36 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-14299-platform-eval-stub-removal branch September 2, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

3 participants