fix(#483): role separation + model-aware prompt budget + stage_summaries schema_version - #513
Conversation
…et, stage_summaries schema_version
The builder prompt embedded the validator's full role file (its rubric —
what it checks, how it judges) via coreAgentContext(), a role-separation
leak the audit named directly. coreAgentContext() now takes a
role: "builder" | "orchestrator" and omits validator.md for the builder;
the orchestrator (which coordinates every role and never renders a verdict)
is unaffected.
priorStageInputsBlock's fixed 3500-char / 6-stage caps are replaced with a
budget computed from the executing model's tier (src/core/harness/
prompt-budget.js) — a documented char approximation, no tokenizer
dependency, keyed to the model_policy tier vocabulary this repo actually
has ('strong' | 'balanced' | 'economy'). A stronger tier now genuinely sees
more prior-stage history; an unknown/missing tier stays conservative.
stage_summaries entries can now carry schema_version; its absence WARNs
(non-blocking, following the #482 evaluateAttemptIdentity precedent) via a
stage_summary_schema_version_missing event, never a hard block.
Deferred scope (documented in docs/HARNESS.md): true dependency-closure
handoff selection (no per-stage dependency graph exists in this codebase),
a real tokenizer, content-hash/snapshot-manifest binding (ties to #482's
own deferred scope), a typed CONTEXT_BUDGET_EXCEEDED blocking result, and a
formal prompt-injection test battery.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Strix Security ReviewNo security issues found. Updated for Reviewed by Strix |
|
Warning Review limit reached
Next review available in: 9 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR Summary by QodoFix role separation, tier-based prompt budget, and stage_summaries schema_version
AI Description
Diagram
High-Level Assessment
Files changed (7)
|
Code Review by Qodo
Context used✅ Compliance rules (platform):
300 rules✅ Skills:
|
…orrect the builder prompt intro Two real findings from Qodo's review of PR #513: - normalizeStageSummaries() reconstructed each stage_summaries entry field-by-field and dropped schema_version, so a compliant builder contract's version was lost the moment it passed through episodic memory — defeating the whole point of the field for anything recalled later. Preserved as a bounded finite number (or null), never a raw pass-through. - The builder prompt's intro sentence still claimed "Follow the embedded orchestrator, builder, validator, and specialist instructions below" after role separation deliberately stopped embedding validator.md for the builder — a real prompt/reality mismatch that could confuse builders about which instructions are actually present. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@strix-security please re-run |
|
Strix is installed on this repository, but we couldn't run this PR security review because this workspace's trial has ended. Add a card to resume code reviews here. |
Summary
Closes the pieces of #483 that are safely shippable this slice (scope adapted to what #482 actually shipped — see the "Honest scope" note in docs/HARNESS.md, this PR's #483 section).
coreAgentContext()used to unconditionally embed the validator's full role file (its rubric) into every caller's context, including the builder's own prompt — the agent being judged could read exactly how it would be judged. Now takesrole: "builder" | "orchestrator"; the builder omitsvalidator.md, the orchestrator (which coordinates every role and never renders a verdict) is unaffected.priorStageInputsBlock's fixed3500-char /6-stage caps are replaced with a budget computed from the executing model's tier (newsrc/core/harness/prompt-budget.js) — documented char approximation (no tokenizer dependency, per the [P0][Durability] Add immutable attempt ledger, BUILT state, leases, CAS transitions, and transactional outbox #481 zero-new-deps precedent), keyed to themodel_policytier vocabulary this repo actually has ('strong' | 'balanced' | 'economy'). A stronger tier sees genuinely more prior-stage history; unknown/missing tiers stay conservative.evaluateAttemptIdentityprecedent) via astage_summary_schema_version_missingevent — never a hard block.Deferred scope (documented in
docs/HARNESS.md): true dependency-closure-based handoff selection (no per-stage dependency graph exists anywhere in this codebase — confirmed before scoping), a real tokenizer, content-hash/snapshot-manifest binding (ties to #482's own deferred scope), a typedCONTEXT_BUDGET_EXCEEDEDblocking result, and a formal prompt-injection test battery.Test plan
npm run typecheck— cleannpm test— 1787/1787 passing (includes new tests:tests/prompt-budget-483.test.js, the [P1][Prompt] Add model-aware prompt budgets and provenance-bound cross-stage handoffs #483 role-separation test intests/builder-prompt-critique-446-451.test.js, theevaluateStageSummarySchemaVersiontest intests/harness-contracts.test.js)npm run lint— 0 errors (4 pre-existing unrelated warnings)node scripts/security-audit.mjs— clean (1 pre-existing tolerated bundled advisory)npm run validate— 196/196 agents passgit diff --check— clean, nopackage-lock.jsondrift🤖 Generated with Claude Code