Skip to content

skills(automation): delete the planned-eval stub and drop its ceiling row (#14296 item 2) - #14578

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-14308-automation-eval-stub-removal
Sep 2, 2026
Merged

skills(automation): delete the planned-eval stub and drop its ceiling row (#14296 item 2)#14578
os-zhuang merged 1 commit into
mainfrom
claude/issue-14308-automation-eval-stub-removal

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Fixes #14308

#14296 item 2 = A, comment 5507419465 (09:27Z, 1A 2A 3A 4A): the planned-eval stubs are deleted and their ceilings step down; real fixtures stay. This is round 2 of #14308 — round 1 (PR #14467, merged) implemented every finding in skills/objectstack-automation/SKILL.md and deferred AUTO-B-04, the evals/README.md planned-eval stub, to this ruling. Per the round-1 ACCEPT comment (5506911275), AUTO-B-04 is the only deferred item recorded for this card, so this round closes it.

Verified before touching anything: evals/README.md's "Planned structure" ASCII tree lists four files that do not exist anywhere in the repo (evals/approvals/test-quorum-behaviors.md, evals/flows/test-schedule-binding.md, evals/flows/test-decision-edges.md, evals/triggers/test-time-relative.md) — it is the planned-structure stub, not a real fixture. skills/objectstack-automation/SKILL.md has zero references to evals/ (grep, 0 hits), so nothing links to the deleted file, and nothing in the repo outside scripts/check-skills-token-ratchet.mjs names its path.

The two-file diff

file change
skills/objectstack-automation/evals/README.md deleted (414 tokens, 43 lines — the template listing planned eval files that do not exist)
scripts/check-skills-token-ratchet.mjs one row dropped: ['skills/objectstack-automation/evals/README.md', 414] — the only edit to that script this flight makes

No prose edit anywhere, no new file, no ceiling raised, no other package touched. The real fixture skills/objectstack-automation/evals/approvals/test-revise-loop.md and its ratchet row (ceiling 1329, currently 550 tokens / headroom 779) are untouched.

Reverse verification — the ratchet red → green pair

Before dropping the row (file deleted via git rm, row still present):

$ node scripts/check-skills-token-ratchet.mjs
✗ check-skills-token-ratchet: cannot read skills/objectstack-automation/evals/README.md — red, not a skip (#4690).

exit 1 — matches the header's documented behaviour exactly: "Missing file or empty read is RED, never a pass (#4690: a gate that cannot find its input must fail, not skip)."

After dropping the row:

$ node scripts/check-skills-token-ratchet.mjs
✓ check-skills-token-ratchet: skills/objectstack-automation/SKILL.md is 12269 tokens (ceiling 12768; headroom 499).
✓ check-skills-token-ratchet: skills/objectstack-automation/evals/approvals/test-revise-loop.md is 550 tokens (ceiling 1329; headroom 779).
✓ check-skills-token-ratchet: skills/objectstack-automation/references/_index.md is 835 tokens — generator-owned (`pnpm --filter @objectstack/spec gen:skill-refs`), measured but not ratcheted.
✓ check-skills-token-ratchet: 36 authored bundle file(s) within their ceilings; 11 generator-owned file(s) measured, not ratcheted.

exit 0 (was 37 authored files before the deletion, 36 after — exactly the dropped row). --self-test: ✓ check-skills-token-ratchet self-test: 64 cases pass. (the row is keyed by exact relative path; nothing else in the self-test's fixtures or pinned counts depends on this row, so dropping it needed no other change).

Gates — head sha 81ec9d3d

Union re-derived after the final commit: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands (checked against this checkout's origin remote — it holds) → 24 command(s), all run under scripts/pm/os-verify-lock.sh (OS_VERIFY_LOCK_SLOT=issue-14308-r2); every exit code captured before any pipe.

command exit result
node scripts/check-ci-filter-parity.mjs 0 all 130 declared cross-package glob(s) (92 unique) covered by core or crosspkg, every crosspkg entry still covers one, and the test job's if: still names both filters
node scripts/check-cross-package-test-inputs.mjs 0 25 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob
node scripts/check-shard-attestation.mjs 0 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s)
node scripts/check-skills-token-ratchet.mjs 0 36 authored bundle file(s) within their ceilings; 11 generator-owned file(s) measured, not ratcheted
node scripts/check-test-completeness.mjs 3 PREREQUISITE NOT MET — no saved turbo run test log named; NOT MEASURED, not a finding
node scripts/pm/bare-root-worklist.mjs --self-test 0 self-test OK — 57 live row(s), 49 recorded verdict(s), none stale/missing/contradicted
pnpm --filter @objectstack/lint run check:doc-formula-expressions 3 PREREQUISITE NOT MET — @objectstack/formula not built; this flight touches neither formula nor its docs
pnpm check:agent-test-spelling 0 0 violations — 430 file(s), 5600 bare -- token(s), 1380 launcher-rooted run(s)
pnpm check:bash32-floor 0 26 tracked shell file(s), no bash4+ construct outside guarded positions
pnpm check:cli-command-ids 0 315 command-id literal(s) across 112 file(s) all resolve
pnpm check:corpus-claim-drift 0 2 baselined file(s), clean
pnpm check:cross-package-test-inputs 0 25 package(s) declared, all covered
pnpm check:doc-authoring 0 82626 string(s) across 1121 parsed source(s) clean; sibling-package baseline holds
pnpm check:entry-guard 0 199 scripts/ file(s), 150 export bindings, all safe
pnpm check:parse-guard 0 198 scripts/ file(s), clean
pnpm check:pm-dispatch-gates 0 dispatch-gates self-test: 1240 cases pass
pnpm check:pm-governed-merges 0 self-test: 243 assertions; live check clean
pnpm check:pnpm-filter-targets 0 142/181 --filter occurrence(s) across 32 file(s) resolve against 78 workspace package(s)
pnpm check:ratchet-remedy-authority 0 184 scripts swept; 12 marked ⛔ MAINTAINER-ONLY, 6 turned down, 166 hand out no remedy
pnpm check:role-word 0 42 baselined file(s) unchanged by this flight
pnpm check:skill-compatibility 0 11 SKILL.md file(s) reconciled against 78 workspace packages
pnpm check:skill-frame-sync 0 4 copies of the decision frame structurally isomorphic across 3 files
pnpm check:skill-identifier-liveness 0 Leg 1: 484 citation(s) over 46 published file(s); Leg 2: 8 registered exhaustive section(s), 0 ledgered gap(s)
pnpm check:watch-hint-literal 0 45 declaration(s) across 4 rostered name(s), no unrostered spelling

check:skill-docs (pnpm --filter @objectstack/spec check:skill-docs) → ✅ Skill docs in sync — no regeneration needed (neither generated file lists evals/). check:skill-compatibility and check:skill-identifier-liveness both green, unaffected by the deletion (the eval README carried no identifier binding: Leg 1 484 citations / Leg 2 8 exhaustive sections, both stable). check:published-readme-links green — 176 outbound links, 103/103 relative targets found (the deleted file was linked from nothing). node scripts/check-nul-bytes.mjs → clean (7953 text files scanned), plus a direct grep -naP C0-control scan of the changed script file, clean.

NOT MEASURED (2): node scripts/check-test-completeness.mjs exits 3, its own PREREQUISITE NOT MET code — needs a saved turbo run test log this run has no reason to produce; CI supplies it. pnpm --filter @objectstack/lint run check:doc-formula-expressions exits 3, PREREQUISITE NOT MET — the workspace package @objectstack/formula is not built; this flight touches neither formula nor its docs, so building it just to green a prerequisite gate would be verification-face padding on a two-file deletion; CI builds the closure and supplies a real reading.

Labels

skip-changeset — this diff releases nothing from any package (skills/** + one scripts/** ratchet-row edit); checked against scripts/check-empty-changeset.mjs's own enumeration (:357, "It releases nothing (.github/, .claude/, skills/, docs/, content/, examples/, tests-only, and the like) -> delete the changeset and apply the 'skip-changeset' label"). No contract claim in this diff ⇒ no needs:contract-review.

Stays draft: skills/** is a governed surface (docs/adr/** · .claude/** · skills/** · AGENTS.md · CLAUDE.md) — landing is the maintainer's, by hand.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1


Generated by Claude Code

… row

#14296 item 2 = A (comment 5507419465): the planned-eval stubs are
deleted and their ceilings step down; real fixtures stay. This is
round 2 of #14308 — round 1 (PR #14467, merged) implemented every
finding in skills/objectstack-automation/SKILL.md and deferred
AUTO-B-04, the evals/README.md planned-structure stub, to this ruling.

Deletes skills/objectstack-automation/evals/README.md (414 tokens —
the template listing planned eval files that do not exist) and drops
its one ceiling row from scripts/check-skills-token-ratchet.mjs. The
real fixture evals/approvals/test-revise-loop.md and its ratchet row
are untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
@os-zhuang
os-zhuang marked this pull request as ready for review September 2, 2026 14:05
@os-zhuang
os-zhuang enabled auto-merge September 2, 2026 14:05
@os-zhuang
os-zhuang added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 07e2457 Sep 2, 2026
36 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-14308-automation-eval-stub-removal branch September 2, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

3 participants