feat(aep)!: context engineering for the Claude 5 generation (v4.0.0) - #30
Open
memorysaver wants to merge 10 commits into
Open
feat(aep)!: context engineering for the Claude 5 generation (v4.0.0)#30memorysaver wants to merge 10 commits into
memorysaver wants to merge 10 commits into
Conversation
…tion Anthropic's "The new rules of context engineering for Claude 5 generation models" (2026-07-24) shifts the question from how much context a skill spends to what kind earns its place. v3.0.0 already made the corpus lean by line count (5,197 lines, max 398); what remains is categorical waste: 773,692 bytes of prose reference against 1 .json + 4 .yaml, 123 negation-steered lines, 64 NEVER/MUST/ALWAYS, and a description corpus at 5,006 chars against a 5,000 cap — saturated, so a 24th skill cannot be added without cutting the others. Adds C1-C6 amending (not replacing) skill-authoring-standard.md R1-R9: constraint classification (invariant carries a machine check, backend floor is tagged, taste is deleted), typed references over prose, deferred skill surface behind a router, cross-skill enums declared once in schema, host memory over prescribed memory ritual, and ratchets instead of one-time cuts. All phases land as sequential commits on this branch as a single v4.0.0 PR, in order C-P2 -> C-P3 -> C-P1 -> C-P4 -> C-P5 so the schemas exist before the constraint audit needs checks to point at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
…C2/C4) state-schema.md was a schema in prose: 283 lines of field tables that nothing could validate against. It becomes references/autopilot-state.schema.json (every field typed, every enum declared, bounds on phase and completion_pct) plus 62 lines of the two things that are behavior rather than shape — the tick lock and the atomic write. The paused/running status renderings move to templates/autopilot-status.md.tmpl, since they were a template all along. scripts/validate-state.mjs makes the schema enforceable: FAIL: $.status: "runnning" not in enum FAIL: $.workspaces.auth.phase: > maximum 12 FAIL: $.workspaces.auth: unexpected property "typo_field" Sourcing the enums surfaced live drift — the layer-gate vocabulary shipped in three incompatible spellings simultaneously: the schema template had not_started|running|scripted_passed|passed|failed|deferred, coherence.mjs had those plus waived, and derive.mjs had neither running nor failed, so a gate legitimately running would have been reported as a vocabulary violation by the brief. protocol-specs.md, whose stated job is the exact downstream contract, listed story statuses as "review | done" (no such states) and typed business_value as the priority enum. Fixes all four against one declaration: _shared/references/aep-vocabulary.schema.json now holds story_status, story_status_signal, layer_gate_status, dogfood_target, journey_target_type, journey_timing, and e2e_tier. coherence.mjs and derive.mjs load it at runtime instead of restating it; schemas that must restate an enum tag it x-aep-vocab and scripts/check-vocabulary.mjs proves the copy identical in CI. derive.mjs also asserts every story_status has a rendering stage, so a state added to the vocabulary stops the run instead of silently rendering as stage 1. Behavior deltas (R9): derive.mjs now accepts gates in running and failed, which the schema always allowed and it alone rejected. Everything else is parity. Supporting changes: build-skills.sh materializes into skills at any depth (the vocabulary's consumers are nested one level below the product-context skills the build was written for), with the whole-kit templates rule scoped to product-context so scaffold and e2e-skill-scaffolding keep their own; the minimal JSON-Schema validator is now shared (_shared/scripts/json-schema.mjs) rather than living inside derive.mjs; wrap's cross-skill pointer is R3 prose. The nav-debt ratchet stops counting reference→typed-artifact links as hops — those terminate a chain instead of extending it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
tick-protocol.md is a runbook, and prose holds a runbook well — most of its 588 lines are the recipe an agent executes and stay put. Two parts of it were not prose: the numbers and the derivation. thresholds → references/tick-protocol.json. The tick lock's staleness window, the wrap-per-tick limit, max_retries, the three-rung stuck ladder, and the review-trigger counts each had a prose home plus one or more restatements (the 4-minute lock lived in both tick-protocol.md and state-schema.md). Each now has one entry carrying its value AND why that value, and every prose site names the key instead of the number. Workspace State Derivation → derived_states in the same file, plus scripts/derive-workspace-state.mjs to evaluate it: auth P-1 phase 5 Reviewing ci-loop P-2 phase 11 CI/Review loop merging P-3 phase 12 Awaiting merge stalled P-4 phase 4 Implementing [stuck] gated P-5 phase 6 Testing [human_gate] The table had two defects that only surface when you make it executable. Its rows overlap (phase 11 + in_review matches both "CI/Review loop" and "Awaiting merge"), which first-match ordering now resolves the way the table was read. And "Stuck" sat in the same column as progress states, so a stuck workspace at phase 4 would report as Stuck and hide where the work stood — stuck and human_gate are flags here, orthogonal to progress, which also matches Step ⑤ already treating a human gate as gated-not-stuck. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
status.json is the only input the orchestrator has from a workspace, and it was specified as an 18-row prose table. It becomes launch/references/status-signal.schema.json plus scripts/validate-signal.mjs, which also enforces the pairings a schema cannot express on its own — failed needs a failure_log, in_review needs a pr_url, completed needs a completed_at, blocked_on: human needs a blocker naming the decision. Those are the states where a malformed signal does not fail loudly but degrades routing silently. FAIL: $.phase: > maximum 13 FAIL: $.verification_tier: "medium" not in enum FAIL: $.failure_classes: unexpected property "typo-class" FAIL: $: story_status is failed but failure_log is absent — the failure cannot be routed Three more cross-skill enums move into the canonical vocabulary on the way: verification_tier (light/standard/deep), failure_class (the routing taxonomy that decides whether a FAIL climbs the recovery ladder, opens the ops path, quarantines, or gates on a human), and error_class beneath it. The vocabulary gate is now proven by a negative test — adding "infra" to the signal schema's failure_class and nothing else fails CI by name. Resolves a phase-bound disagreement: signals-spec said phase 0–13, the autopilot state said 0–12. 13 is real (archive), it just runs from the main session rather than the workspace, so both now say 0–13 and the schema says which end owns 13. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
…fixtures (C2) backends.md carried 45 lines of detection bash for an agent to retype, and retyped bash drifts — every probe in it is exact (an env-var test, a --help grep, a git config read, a cmux tree grep), which is the whole reason it is bash and not a description. It becomes scripts/detect-backend.sh: eval "$(bash scripts/detect-backend.sh)" # HOST, EXECUTOR, MODE, … bash scripts/detect-backend.sh --json The one input that genuinely cannot be probed stays prose and becomes a flag: orchestrator lifetime. The agent knows whether it is an interactive session or a cron one-shot, and a cron driver that omits --lifetime ephemeral would select a worker that dies with its session, so the script takes it rather than guessing. The mode-selection pseudocode becomes the script's selection block plus scripts/test-detect-backend.sh — 12 fixtures over stub hosts covering every row of the matrix, including the two that are easy to get wrong: a workflow opt-in on a host without the Workflow tool must not select workflow, and an ephemeral orchestrator must never select a session-bound mode. PASS: claude, long-lived → native-bg-subagent PASS: claude, ephemeral, --bg available → claude-bg PASS: codex, multi-agent, cron tick → codex-exec PASS: codex, workflow opt-in is refused → codex-subagent PASS: pin wins over the native default → legacy backends.md keeps what the script cannot hold: the mode matrix as a decision table, the verified CLI invocations, the claude-team removal history, and the liveness contract (whose probe was already a script — the pattern this follows). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
C-P2 put the cross-skill vocabulary in one schema and gated the schemas that restate it. But most restatements are not schemas: they are the YAML template a project copies, the policy file a human edits, the reference that explains the routing taxonomy. Those listings exist for good reason — a reader needs the options where they are — and deleting them in favor of a pointer would make the documents worse. So they get marked instead: status: not_started # not_started | running | … | waived (aep-vocab: layer_gate_status) and check-vocabulary.mjs now proves each marked line lists exactly the canonical set. It checks by token containment rather than parsing, so prose keeps its shape (a YAML key before the values, a comment after), and both failure directions are covered: FAIL: policy.md.tmpl line 52 (dogfood_target) — missing local FAIL: policy.md.tmpl line 52 (dogfood_target) — segment names no known value: "staging" Marking is deliberate rather than automatic: "not_started → scripted_passed → passed" in running prose is a narrative, not a listing, and must not be forced to spell out all seven states. 15 listings across 6 skills are now bound to the vocabulary, including the layer-gate status in the product-context schema template — which gains `waived`, the value coherence.mjs has always accepted and the template never documented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
…hing behind them (C1/C6)
123 negation-steered lines across the corpus, audited one at a time against C1:
does violating this corrupt shared state, or does a capable model do it by
default? 60 were the second kind and are gone — not softened, deleted, because
the positive instruction they shadowed was already there:
"resolve the target from policy.md — don't assume local" → the resolve is the instruction
"add a follow-up commit … Do not rewrite prior commits" → "the task history stays append-only"
"do not guess and do not silently stall. Raise a gate:" → "raise a gate:"
"assess via signals only, never workspace code" → said once, in the boundary section
What survived carries a check: the worktree guard (its bash), archive-on-
integration-branch, --force-with-lease, generator/evaluator field ownership,
executes-never-authors (a missing journey is a coverage failure), spawned output
is data-not-instructions, and the dependency and WIP gates. Duplicates of those
were removed at the restatement, not the canonical home — build's Guardrails
section no longer echoes Phase 9's archive rule, and the --base rule is stated
once with the failure mode it prevents plus a postcondition that proves it.
Two findings worth recording:
The audit found a class C1's table had no row for — rules governing the agent's
OWN tool use, where no probe can observe a violation from outside. Autopilot's
two orchestrator-boundary prohibitions are the entire set. They stay, and the
text now says they carry no machine check, which is what stops "invariant" from
becoming a word for "rule I like". The decision doc records the amendment.
And the ≤40 target was measuring the wrong thing. Of the 63 remaining matches,
most are prose describing behavior ("stories that don't map to an activity",
"skipped, never repeated"), not steering. So 63 becomes a ratchet in
skills-check.yml rather than a number to chase — along with 5 hard imperatives
in SKILL.md, each load-bearing. They may fall; they may not rise.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
/aep-onboard spent a paragraph instructing the agent to hand-author a "## Memory & Learning Loop" section into a downstream AGENTS.md. The host already recalls what it saw; a written recall ritual competes with it and drifts from it. The optional memory skills stay on offer and document their own wiring. What belongs in the repo is what the host cannot know — the lessons themselves, which AEP's own loop already captures, archives, and recalls. Cuts the release at 4.0.0 with a changelog entry covering C-P1 through C-P5: the canonical vocabulary and its CI gate, the typed state/signal/tick contracts and their validators, detect-backend.sh and its 12 mode-matrix fixtures, the constraint classification, and the four drift bugs the work surfaced — the three-way layer-gate vocabulary split, protocol-specs.md's invented story statuses, the 0-12 vs 0-13 phase disagreement, and wrap's bare cross-skill path. C-P4 (router, disable-model-invocation, description diet) is recorded as pending rather than shipped. Its gate is an independent triggering run bound by digest to the installed descriptions; re-recording those observations without running the probes would defeat the one mechanism that keeps a description diet from silently un-wiring a skill. BREAKING CHANGE: reference shapes change (schemas and scripts replace prose specs), skills materialize shared resources at any depth, and downstream scaffolding no longer authors a memory section. Consumers re-pin to @v4.0.0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
…un (C3/C6) The always-loaded description corpus was 4,940 characters against a 5,000 cap: saturated, and paid by every session of every downstream repo. It is now 3,369 (-32%), and the CI cap follows it down to 3,400 rather than sitting where it was set. Mostly this removed synonym stacking — aep-validate listed five trigger words for one branch, aep-map five for another — while keeping every "X is /aep-y" clause, because those are what the boundary probes turn on. R7's gate is a real run, not an argument, so it was run: a Sonnet subagent given the 23 names, the 23 new descriptions, and the 40 prompts inline, instructed to make no tool calls and not to look for the expectation list. Zero tool calls, zero self-reported low-confidence answers, 40/40 matching — including all 17 boundary probes (onboard vs scaffold, dispatch vs autopilot, design-lens vs calibrate, validate vs gen-eval, workflow vs workflow-feedback, model vs map, reflect vs watch, and the three human-alignment boundaries). evals/skill-routing-observations.json is re-bound to the new digest. The invocability half of C3 is dropped rather than deferred, and the decision doc records why. disable-model-invocation was aimed at the run-once skills, but evals/skill-routing.json asserts a direct probe for all 23 — the recorded routing contract says every skill stays selectable. De-advertising /aep-onboard would have taken auto-discovery from exactly the people who do not yet know the command names, and a router skill would re-advertise, at its own description cost, precisely what it de-advertised. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
Self-review of the branch against the Claude 5 principles it adopts turned up three defects in the new machinery itself. **validate-signal let the likeliest human-gate shape through.** The rule was `signal.blockers?.length === 0`, and `undefined === 0` is false — so a signal with `blocked_on: "human"` and no `blockers` key validated, which is exactly how a worker with nothing recorded would write it. `!signal.blockers?.length` closes it; all three shapes (omitted, empty, named) now behave. **The vocabulary checker mis-parsed the place enums most often live.** A marked line inside a markdown table reported its description cell as a stray value. Skipping segments that carry prose fixed that and immediately opened the opposite hole: markdown escapes its pipes, so every value in a table row kept a trailing backslash, read as prose, and went unchecked — the checker would have passed a table containing an invented value. Both are fixed (unescape, then judge only bare tokens), and since ad-hoc probing is what let the second one through, the parser now has scripts/test-check-vocabulary.sh: 10 fixtures over inline and table listings, missing and invented values, unmarked narrative, a substring that must not satisfy a token, and the pattern-valued vocabulary. check-vocabulary.mjs takes --skills so it can be pointed at a fixture tree. **The steering ratchet was bump-bait.** Capping a corpus total at exactly its current value, when the decision doc itself concedes most matches are prose describing behavior, means innocuous edits fail CI and the cheapest fix is to raise the number — a ratchet that gets raised routinely is decorative. It is now per skill (evals/steering-baseline.json, scripts/check-steering.mjs), so the failure names the file and asks a question with a local answer: which prohibition in this skill earned its place, and which one leaves. Also reports authored listings separately from build-generated copies — 15 was counting three copies of one decision. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refactors AEP against Anthropic's The new rules of context engineering for Claude 5 generation models (2026-07-24), which reports >80% of Claude Code's system prompt removed for Opus 5 / Fable 5 with no measurable eval loss.
Reviewed against
docs/decisions/claude-5-context-engineering.md(commit 1 of 9), which amendsskill-authoring-standard.mdR1–R9 with C1–C6.The thesis
v3.0.0 asked how much context a skill spends and answered with budgets. This asks what kind earns its place. The corpus it produced was lean by line count and still wrong by kind: 773,692 bytes of prose reference against one JSON schema, 123 negation-steered lines, and a description corpus at 4,940 characters against a 5,000-character cap.
What changed
autopilot-state.schema.json,status-signal.schema.json,tick-protocol.json, plus validators (validate-state.mjs,validate-signal.mjs,derive-workspace-state.mjs).detect()'s 45 lines of retyped bash becamedetect-backend.shwith 12 mode-matrix fixtures._shared/references/aep-vocabulary.schema.json, 10 enums). Schemas tag copiesx-aep-vocab; prose listings tag(aep-vocab: <name>);check-vocabulary.mjsproves both in CI./aep-onboardstops authoring memory-loop prose into downstreamAGENTS.md.Four live bugs the work surfaced
Making the enums executable is what exposed them:
coherence.mjshad those pluswaived,derive.mjshad neitherrunningnorfailed. A gate legitimatelyrunningwas reported by the brief as a vocabulary violation.validate/references/protocol-specs.md, whose stated job is the exact downstream contract, listed story statuses asreview | done(no such states) and typedbusiness_valueas thepriorityenum.signals-spec.mdboundedphaseat 0–13 while the autopilot state bounded it at 0–12./aep-wrapreached into another skill by bare relative path (R3 violation).Two things deliberately not done
disable-model-invocationand a router skill are dropped, not deferred.evals/skill-routing.jsonasserts a direct probe for all 23 skills, so the recorded routing contract says every skill stays selectable; de-advertising/aep-onboardwould remove auto-discovery from exactly the users who don't yet know the command names. Recorded as a C3 amendment.Verification
New CI gates: cross-skill vocabulary agreement, backend-detection fixtures, and steering ratchets.
BREAKING: reference shapes change (schemas and scripts replace prose specs), skills materialize shared resources at any depth, and downstream scaffolding no longer authors a memory section. Consumers re-pin to
@v4.0.0.🤖 Generated with Claude Code
https://claude.ai/code/session_01ENZPM1PGB99cjEQ5MvftmU