Skip to content

skills(automation): optimization flight — the authored control-flow trio taught and the BPMN trio demoted, notify given its fence, the eval deduplicated, hooks and ETL pointed away (net −1,278 tokens) - #14467

Merged
os-zhuang merged 3 commits into
mainfrom
claude/issue-14308-skills-automation-optimization
Sep 2, 2026
Merged

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Part of #14308

Skills optimization flight for skills/objectstack-automation, implementing the audit record posted on that card (25 findings, read-only audit at a59f78d). Shrink-only under the token ratchet: every addition is paid by deletions in the same file, no ceiling was raised, no file added or deleted, and scripts/check-skills-token-ratchet.mjs was not touched.

Head sha for every gate reading below: 2926abc9d518b61220ba9dd2e288607cdef43247.

Per-file token delta

file before after delta ceiling headroom after
skills/objectstack-automation/SKILL.md 12,768 12,269 −499 12,768 499
skills/objectstack-automation/evals/approvals/test-revise-loop.md 1,329 550 −779 1,329 779
package authored total 14,511 13,233 −1,278

Lines, the primary reading for a skills/** diff: SKILL.md 1,001 → 949 (−52); the eval 81 → 37 (−44). Whole published bundle, all 11 SKILL.md files: 10,183 → 10,131 lines, 114,923 → 114,424 tokens (−499). evals/README.md (414) and the generated references/_index.md (835) are untouched.

skills/README.md and content/docs/ai/skills-reference.mdx are generator output onlypnpm --filter @objectstack/spec gen:skill-docs, forced by the description edit; their diff is the two description lines and nothing else.

Per-item — 落点 | before | after

Byte deltas are exact per-edit measurements; token figures are ceil(bytes/4), the ratchet's own convention. Composite rows are marked.

The two falsehoods

id 落点 before after Δ bytes
AUTO-A-01 intro :22-25 + When-to-Use :35 "This skill covers … and ETL pipelines" / "You are designing an ETL pipeline" both deleted. The L2 ETL layer was removed at spec 17 with no replacement (packages/spec/src/migrations/entries/semantic/17.etl-pipeline-layer-retired.ts:7-20) and the package declares 17.x. grep -n ETL SKILL.md is now 0. −300
AUTO-F-01 node table :76-79, BP :791, pitfall :828 "Always handle errors. Add boundary_event nodes" — a form with 0 authorings the showcase's own waiver names the author-facing equivalents (examples/app-showcase/src/coverage.ts:299-300); the advice is gone and the trio now has exactly one row, marking it BPMN-interop lowering (never deleted the last mention, per the ruling) see F-01 below

Funded additions — every claim verified at source

id 落点 before after Δ bytes source
AUTO-F-01 #### Control Flow table 3 gateway rows; parallel/try_catch absent; loop = one clause rows for loop (config.body, iteratorVariable default item, indexVariable, maxIterations), parallel (config.branches[], ≥ 2, implicit join), try_catch (config.try/catch, errorVariable default $error, config.retry, no finally); the trio collapsed into one "not author-facing" row +556 (+139 tok) packages/spec/src/automation/control-flow.zod.ts:41-46 (the three type ids vs the "author-invisible interchange representations"), :163-219 LoopConfigSchema, :257-268 ParallelConfigSchema (.min(2)), :301-320 TryCatchConfigSchema incl. the finally guidance string
AUTO-F-02 ### Prerequisite prose + fence naming automation/triggers/job/queue 7-row capability → surface table adding approvals and messaging, each with what its absence does +399 (+100 tok) packages/spec/src/kernel/platform-capabilities.ts:31-66 (all 7 tokens are canonical), examples/app-showcase/objectstack.config.ts:92-107 (the inline rationale for approvals / messaging), packages/services/service-automation/src/builtin/notify-node.ts:313-325
AUTO-E-04 :97, the whole coverage of notify one table cell, no config keys a ### notify section with the 8-key fence, plus the RAW-vs-interpolated split marked inline +996 (+249 tok) packages/spec/src/automation/io-node-config.zod.ts:178-258re-located by content: NotifyConfigSchema spans 178-258 on this base, not the audit's :179-218. severity is a closed z.enum(['info','warning','critical']) at :242; the interpolate/RAW split is the module note at :160-168; the sourceObject/sourceId pair-or-dropped rule at :169-172
AUTO-E-04 Valid-but-silently-wrong absent item 5: notify returns success with output: { delivered: 0, failed: 0, skipped: true } and metrics.acted: 0 when no messaging service is registered +330 (+83 tok) notify-node.ts:313-325 verbatim
AUTO-G-02 Flow Types record_change row the two spellings never said to coexist one clause: autolaunched + a record-* start binding behaves identically; record_change additionally opts into the trigger-readiness lint +172 (+43 tok) packages/services/service-automation/src/engine.ts resolveTriggerBinding; packages/lint/src/validate-flow-trigger-readiness.ts keys on record_change
AUTO-E-01 schedule-flow example 1 of 13 TS fences carried the os:check marker the schedule flow marked and made self-contained (import { defineFlow }, status: 'active') — the gate went 265 → 266 blocks +138 (+35 tok) packages/spec/scripts/check-skill-examples.ts:18-23
AUTO-A-02 / A-03 frontmatter description "scheduled jobs, and webhooks"; no screen-flow routing names the jobs (defineJob) / webhooks (defineWebhook) stack collections and the screen-flow / wizard route objectstack-ui points here +132 (+33 tok) packages/spec/src/system/job.zod.ts:170, packages/spec/src/automation/webhook.zod.ts:245, packages/spec/src/stack.zod.ts:459 + :523; skills/objectstack-ui/SKILL.md:11-13
flight scope References grep -n evals SKILL.md → 0 hits, the eval tree was unreachable one link to evals/approvals/test-revise-loop.md +120 (+30 tok)

The two rules ported from the query flight

The query flight (#14443) deleted 21 lines of flow-filter text on the ruling that this package is the anchor, and found two rules the anchor lacked. Both are backed at source and both landed, in a new ### Filter tokens (config.filter) heading — which is also the addressable anchor that ruling asks this package to keep (it was :157-164 on base, inside an unnamed blockquote; it now has a heading of its own).

rule what landed source
(a) precedence on config.filter The flow template engine runs first. A whole-string token it resolves is a flow value; one it does not resolve that IS a recognised filter placeholder ({current_user_id}, {current_year_start}) passes through verbatim for the query engine to expand — so a flow variable named after a placeholder shadows it. Only filter gets the hand-off; title/message/fields/url keep plain interpolation. packages/services/service-automation/src/builtin/template.ts:368-411interpolateFilter, incl. the whole-string regex /^\{([^{}]+)\}$/ and the comment "Flow variables keep precedence — only an unresolved token is considered for hand-off". isKnownFilterToken is packages/spec/src/data/context-tokens.zod.ts:219.
(b) the static-checkability split A {record.…} token inside a filter naming an unknown field, or hopping a relation the start node does not list in config.expand, is an ERROR at objectstack validate (the condition is DROPPED and the node refuses to execute). The same reference outside a filter is a warning. A {var} naming a flow variable or node output is not statically checkable at all. packages/lint/src/validate-flow-template-paths.ts:36-50 (severity follows the runtime consequence, by position), :403-424 (unknown field, severity: inFilter ? 'error' : 'warning'), :427-456 (relation hop, same split, config.expand opt-in at :276-290), :59-62 (only record.-prefixed tokens are checked — the third clause).

Cost: +1,226 bytes (+307 tok), paid inside SKILL.md. This is the amount the card's target explicitly excludes ("minus the two ported rules below").

Deletions and merges

id 落点 before after Δ bytes
AUTO-B-01 Valid-but-silently-wrong at 84% depth 1,630 tok of the highest-decision-density content began at line 836 promoted to a top-level section immediately after Flows; items renumbered 1-5 as their own list, the stale "see pitfall 9" cross-reference re-pointed −87 (move)
AUTO-B-02 ### Flow Example run-up 893 tok of blockquotes between the heading and the example only the blockquote about the code that follows stays adjacent; the rest moved below the fence under ### Failure routing & runAs ~0 (move)
AUTO-B-03 ### State Machine Design BPs 5 items, 4 restating the Notes block 520 lines earlier subsection deleted; the one new item (an unlisted from state is not guarded) merged into that Notes block composite, see below
AUTO-B-06 pitfall 8 vs Time-relative section both opened with the same rule the restated half deleted; the unique half (date equality never matches + the abutting-window construct) folded into the Time-relative section −373
AUTO-B-07 ## Verify your work third statement of pitfalls 6 and 10 both paragraphs deleted; the os validate / os build block and the one clause nothing else carries (a bare ref resolves, so a typo there is advisory) kept −519
AUTO-C-01 / AUTO-G-01 :37-43 CEL stdlib enumerated, then deferred to formula; two tagged-template spellings endorsed with no rule enumeration deleted (pointer kept); one sentence says a slot takes a plain CEL string, the tag is the same string with author-time validation — pick one per file −23
AUTO-C-02 cheat sheet :596, :600-606 2 object-hook rows + an 8-line ctx.record paragraph rows and paragraph deleted, one-line pointer to objectstack-data references/data-hooks.md −649
AUTO-C-03 CRM Blueprint row 4 action-to-flow bridge restated deleted −138
AUTO-C-04 :731-734 4th copy of the OLD/NEW retirement parenthetical deleted; the formula link was already on the next line −86
AUTO-D-01 fault-edge blockquote :152-164 212 tok of justification prose a two-cell ROUTES / does-not-route row plus the one consequence sentence −229
AUTO-D-02 :404-409 ADR-0044 amendment history deleted; the decision it changes is still stated twice −239
AUTO-D-03 ### Re-homing the old process model a 6-row pre-ADR-0019 migration table deleted; every destination is taught under "Branching, side-effects & rejection", and the one fact the table alone carried (there is no approvals stack collection) moved into the surviving blockquote −921
AUTO-D-04 :895-902 spec-17 retirement narration deleted; the positive prescription kept −426
AUTO-D-05 :926-941 215 tok of rationale after the rule and its OK/NO pair the defineStack({ functions }) fence kept with a one-line caption −399
AUTO-D-06 ### Flow Design BPs 5 generic items; #5 duplicated an in-example comment; #2 was falsehood 2 subsection deleted composite
AUTO-D-07 pitfalls 1-5 engine-implementer advice, BPMN interop, a restated idempotency rule #1/#2/#5 deleted; #3 (end node) folded into the flow-example blockquote, #4 into the trigger blockquote −503
AUTO-D-08 :22-25 intro paragraph restating the frontmatter deleted; the When-to-Use bullets kept −234
AUTO-D-09 ### Trigger Design BPs 3 items, 2 restating the start-node condition text #2 merged into the trigger section as a blockquote composite
AUTO-E-02 trigger-config example read as a whole flow (name/label/type/nodes) but omitted edges, which FlowSchema requires rewritten as the start-node fragment it actually is, with a sentence saying so −54

Composite row: AUTO-D-06 + AUTO-B-03 + AUTO-D-09 were one contiguous ## Best Practices section; deleting all three subsections removed it entirely — −1,648 bytes, against +230 re-added for B-03's surviving item and +248 for the D-09/D-07 survivors.

premise_false

  • premise_false: AUTO-A-01 (description half) — the card says the falsehood is also in "the frontmatter/description". It is not: the description on base reads "Flows (visual logic), Triggers, Approvals, state machines, scheduled jobs, and webhooks" and never mentions ETL. grep -n ETL SKILL.md on base returns exactly lines 24 and 35, both body. Nothing was deleted from the description on this account; the description edit that did land is AUTO-A-02/A-03.
  • premise_false: AUTO-E-04 spanio-node-config.zod.ts:179-218 does not bound NotifyConfigSchema on this base; the schema is :178-258 and three of the eight documented keys (severity, sourceObject/sourceId, actionUrl) sit past :218. Re-located by content, as instructed; every key was read at its real line.

Everything else in the flight scope had a true premise. Two findings were not implemented, by constraint rather than by premise:

os:check marking — what is and is not claimed

Per the ruling, only blocks that compile against @objectstack/spec declarations are marked.

  • Marked (AUTO-E-01): the schedule-flow example. check:skill-examples counts 265 → 266 marked blocks.
  • Not marked, and why: the new notify config fence is a config: { … } fragment — the shape an author copies into a node — and is not self-contained. Making it compile would mean wrapping it in a whole defineFlow, which would teach a different thing and cost roughly triple. The audit's other candidate, the record-change trigger example, was deliberately turned into a node-only fragment by AUTO-E-02, which is incompatible with marking it; the copy hazard AUTO-E-02 names is removed instead.

Reverse verification of the new marker

To prove the new marker is not a phantom green, on the committed tree: mutated type: 'schedule' to type: 'schedulez' inside the marked block, confirmed the mutation reached disk by blob hash (8824aa3f…0019f5e2…), and check:skill-examples went red with error TS2820: Type '"schedulez"' is not assignable to type '"api" | "screen" | "autolaunched" | "record_change" | "schedule"'. Restored with git checkout HEAD -- the absolute path under an EXIT/INT/TERM trap; restoration proven by blob hash back to 8824aa3f…, git diff HEAD empty, and 0 occurrences of the injected token. The restore leg was re-measured: 266 blocks, exit 0. No package build is involved — this gate reads the skill file directly, so there is no dist/ leg to preflight.

follow-up for other packages

Anchors were re-verified at this head; all are equal or stronger than the text deleted here, so nothing is owed except the last item.

  • objectstack-datareferences/data-hooks.md:346 carries the ctx table AUTO-C-02 deleted here, plus at :358 the nuance this file omitted ("Action bodies additionally receive ctx.recordId and ctx.record"). Stronger. Path kept by the delivered data flight. Nothing owed.
  • objectstack-formulaSKILL.md:160-172 is the stdlib table (AUTO-C-01), :383 and :399 the OLD/NEW translation row and its warning (AUTO-C-04). skills/README.md:87 carries the same. Stronger. Nothing owed.
  • objectstack-uiSKILL.md:905 is the os:check-marked defineAction({ type: 'flow' … }) example (AUTO-C-03). Type-checked, so stronger. Nothing owed.
  • objectstack-upgradeSKILL.md:82-93 carries os migrate meta --from 16 and its variants (AUTO-D-04). Stronger. Owed: it carries no pre-ADR-0019 ApprovalProcess.create({…}) re-home table, so AUTO-D-03's content now has no home in any package. The audit's own reasoning is that every destination is taught in this file under "Branching, side-effects & rejection", so this is recorded as a judgement to confirm, not a regression to fix — a follow-up for objectstack-upgrade if a maintainer wants the cross-major mapping preserved.

Out-of-scope card filed

#14466[finding] the objectstack-automation reference index routes to the ADR-0020-retired state-machine.zod.ts and omits both per-node config schemas (SKILL_MAP in packages/spec/scripts/build-skill-references.ts:120-129). This is audit id AUTO-B-05, deferred by this card's ruling because the fix lives in packages/spec. Unassigned, finding label, no pm-state, no priority. Deduped first against a healthy in-session search (a known-hit control query returned this card, so an empty result would have been a real reading); the nearest neighbour is #14462, the same defect class for objectstack-ai — worth triaging together, possibly one fix.

Gap against the card's target

The card targets ≈ −1,125 net in SKILL.md, "minus the two ported rules". Delivered −499. Itemised rather than smoothed:

tok
card target, net −1,125
the two ported flow-filter rules (explicitly excluded by the target) +307
adjusted target −818
delivered −499
gap 319

Where the gap sits, measured against the card's own funded allowances: AUTO-E-04 cost +332 tok against +140 funded (+192 over) and AUTO-F-02 +100 against +40 (+60 over). Both overruns are verified contract content, not prose: the notify RAW-vs-interpolated split and the per-capability "absent ⇒" column. Two trim passes brought them down from +359 and +143. The remaining ~67 tok is spread across merge survivors (AUTO-B-03, AUTO-D-09/AUTO-D-07) that the audit costed at 0 and that landed as short blockquotes instead. Nothing was cut from the audit's delete list to close it, and no verified fact was dropped to reach a number.

The eval over-delivered in the other direction: −779 against −700 funded.

Gates

Every exit code captured before any pipe, at head 2926abc9. The full family was re-derived after regenerationnode scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, which reads the change set from git itself (4 paths vs merge base a98b61b3e) — and all 36 commands it printed were run.

  • 35 green, exit 0. Including: node scripts/check-skills-token-ratchet.mjs; pnpm --filter @objectstack/spec check:skill-examples (266 blocks, run after building the @objectstack/spec and @objectstack/client-react closures so it could not false-green on a stale dist/); pnpm --filter @objectstack/spec check:skill-docs; pnpm check:skill-compatibility; pnpm check:skill-identifier-liveness; pnpm check:skill-frame-sync; pnpm check:doc-authoring; pnpm check:doc-anchors; pnpm check:published-readme-links; pnpm check:corpus-claim-drift; the 20 docs-family gates the .mdx pulls in.
  • 1 NOT MEASURED: node scripts/check-test-completeness.mjs exits 3 with PREREQUISITE NOT MET — it grades a saved turbo run test log, and the derived family names it with no argument. Its own text says this branch "is not a red, and there is nothing here to fix". Recorded as not measured, not as green.
  • node scripts/check-nul-bytes.mjs green (7,902 files, no raw control bytes); the four changed files also scanned directly with grep -naP for the C0 range — clean.
  • Baselines were taken on origin/main before the first edit and were green, so every reading above is a delta against a known-green base.

Liveness moved in the right direction: Leg 1 went 490 → 495 citations checked against the implementation word tokens, i.e. the five new identifiers this PR writes are all live. Leg 2 has no binding on this package (BINDINGS in scripts/check-skill-identifier-liveness.mjs:284-350 registers 8 sections, none in objectstack-automation), so the node-type table is not registered exhaustive and the gateway-row rewrite could not silently unbind an assertion — checked before the first deletion, as instructed.

Labels and landing form

skip-changeset applies and is verified against the gate's own enumeration, not a remembered list: scripts/check-empty-changeset.mjs:357-359 names ".github/, .claude/, skills/, docs/, content/, examples/, tests-only, and the like" as the releases-nothing route, and the diff is exactly skills/** plus one generated content/docs/** file. needs:contract-review is applied to both carriers — the ADR-0031 control-flow rows, the notify skipped: true semantics, the capability table and the two ported flow-filter rules are all contract claims.

Governed face, so this stays draft. The card stays open on purpose: AUTO-B-04 (the evals/README.md planned-structure stub) is deferred to #14296 item 2, which is why the first line is Part of and no closing keyword appears anywhere in this body.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1


Generated by Claude Code

The eval restated SKILL.md's "Send-back for revision (ADR-0044)" section rather
than testing beyond it: the canonical shape, the four lint findings, the
wait-vs-approval_revise rationale and the showcase pointer all had a second copy
here. 2,204 tokens -- 15% of the authored package -- went to a surface with one
real authoring in the whole corpus, while `notify` (20 authorings) got 15.

Delete the Expected-Output fence and the Common-Mistakes rows that restate the
skill; keep Scenario + Validation Criteria, which is what a grader needs. The one
row that did NOT restate SKILL.md (a "revise mode" re-suspend of the approval
node) survives as criterion 7, so no graded fact is lost.

1,329 -> 550 tokens (ceiling 1,329).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
…BPMN one

The package taught only `parallel_gateway` / `join_gateway` / `boundary_event`
-- 0 authorings anywhere in the corpus -- and never mentioned the ADR-0031
structured constructs the corpus actually uses (`loop` 3, `try_catch` 2,
`parallel` 2). `notify`, the most-used node type at 20 authorings, had one table
cell and no config keys, including no mention of its documented silent no-op.

ADD (funded, each claim verified at source):
  AUTO-F-01  `parallel` / `try_catch` / `loop`-container rows; the gateway trio
             marked BPMN-interop lowering, not author-facing.
  AUTO-F-02  capability -> surface table, adding `approvals` and `messaging`.
  AUTO-E-04  `notify` config fence + the messaging-absent silent success.
  ported     flow-filter precedence + the static-checkability split (this
             package is the anchor; the query flight deleted its copy).
  AUTO-G-02  `autolaunched` + a `record-*` binding behaves like `record_change`.
  AUTO-E-01  os:check marker on the schedule-flow example (265 -> 266 blocks).
  AUTO-A-02/A-03  `jobs` / `webhooks` collections and screen-flow routing in the
             description.

DELETE / MERGE: AUTO-A-01 (ETL, retired at spec 17), AUTO-B-03, AUTO-B-06,
AUTO-B-07, AUTO-C-01, AUTO-C-02, AUTO-C-03, AUTO-C-04, AUTO-D-01..D-09,
AUTO-E-02. REORDER (free): AUTO-B-01, AUTO-B-02.

12,768 -> 12,269 tokens (ceiling 12,768; every addition paid in this file).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
Generator output only -- `pnpm --filter @objectstack/spec gen:skill-docs`.
`check:skill-docs` is a drift gate, so the AUTO-A-01/A-02/A-03 description edit
forces both files.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation labels Sep 2, 2026
@os-litant os-litant added skip-changeset PR has no user-facing published change; bypasses the changeset gate needs:contract-review and removed documentation Improvements or additions to documentation labels Sep 2, 2026 — with Claude
@os-zhuang
os-zhuang marked this pull request as ready for review September 2, 2026 11:24
@os-zhuang
os-zhuang enabled auto-merge September 2, 2026 11:24
@os-zhuang
os-zhuang added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 9d755ba Sep 2, 2026
47 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-14308-skills-automation-optimization branch September 2, 2026 12:23
os-litant pushed a commit that referenced this pull request Sep 2, 2026
… row

#14296 item 2 = A (comment 5507419465): the planned-eval stubs are
deleted and their ceilings step down; real fixtures stay. This is
round 2 of #14308 — round 1 (PR #14467, merged) implemented every
finding in skills/objectstack-automation/SKILL.md and deferred
AUTO-B-04, the evals/README.md planned-structure stub, to this ruling.

Deletes skills/objectstack-automation/evals/README.md (414 tokens —
the template listing planned eval files that do not exist) and drops
its one ceiling row from scripts/check-skills-token-ratchet.mjs. The
real fixture evals/approvals/test-revise-loop.md and its ratchet row
are untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LraLgQVGq8egUwfYZpbYt1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants