Skip to content

Move reviseRubric into the adversary config and factor round boilerplate - #21

Merged
Tyler-R-Kendrick merged 2 commits into
mainfrom
claude/harness-config-boilerplate-28r6r4
Jul 12, 2026
Merged

Move reviseRubric into the adversary config and factor round boilerplate#21
Tyler-R-Kendrick merged 2 commits into
mainfrom
claude/harness-config-boilerplate-28r6r4

Conversation

@Tyler-R-Kendrick

Copy link
Copy Markdown
Owner

What changed

reviseRubric now belongs to the adversary

Revising the rubric after a standing challenge is the adversary's sole responsibility, so it no longer sits as a stand-alone option on HarnessInput. The adversary option is now an AdversaryConfig:

adversary: {
  challenge: myAdversary,            // required
  reviseRubric: myRubricRevision,    // optional; defaults to appending the challenge evidence as new criteria
}
  • The rubric revision is dispatched on the bus as the adversary actor (previously teacher), and the "must improve the rubric" error message names the adversary accordingly.
  • New exported types: AdversaryConfig, AdversaryTurn, RubricRevisionTurn — replacing the inline Readonly<{ ... }> turn shapes.
  • The createHarnessLoop provider, tests, README, and architecture doc are updated to the nested config.

Factory helpers for the boilerplate object initialization

The run loop rebuilt the same frozen shapes over and over; those now go through small factories:

  • signal — the ...(input.signal === undefined ? {} : { signal: input.signal }) spread, computed once per run instead of three times.
  • contextOf(actor?) — provider-shaped bus history, replacing repeated provide(await bus.read(...)).
  • evidence — the shared fields both judge verdict requests carry.
  • record(challenged?) — one factory for every recorded round, replacing four near-identical rounds.push(Object.freeze({ ... })) blocks.
  • The provider adapter gets the same treatment with a local maybeSignal helper.

Verification

  • npm run typecheck — clean across all packages.
  • npm test — 81 tests in 15 files, all passing.

🤖 Generated with Claude Code

https://claude.ai/code/session_0175hpwygwMcHwR4xWFVgwST


Generated by Claude Code

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@Tyler-R-Kendrick, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 15e2414c-26c9-4706-ac8d-75971b6169c3

📥 Commits

Reviewing files that changed from the base of the PR and between d5fc664 and abe0881.

📒 Files selected for processing (6)
  • docs/architecture.md
  • packages/harness/README.md
  • packages/harness/src/harness.ts
  • packages/harness/src/index.ts
  • packages/harness/test/harness.test.ts
  • src/providers/harness.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/harness-config-boilerplate-28r6r4

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Rubric revision after a standing challenge is the adversary's sole
responsibility, so HarnessInput.adversary is now an AdversaryConfig — a
required challenge callback plus an optional reviseRubric callback — and
the top-level reviseRubric option is gone. The revision is dispatched on
the bus as the adversary actor, and the named AdversaryTurn and
RubricRevisionTurn types replace the inline turn shapes.

The run loop's repeated object initialization now goes through small
factories: a once-computed optional-signal spread, a contextOf(actor)
helper for provider-shaped bus history, shared verdict-request evidence,
and a record() factory replacing four near-identical rounds.push blocks.
The provider adapter gains the same maybeSignal helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hpwygwMcHwR4xWFVgwST
@Tyler-R-Kendrick
Tyler-R-Kendrick force-pushed the claude/harness-config-boilerplate-28r6r4 branch from f6d3bea to 81c01dc Compare July 12, 2026 21:18
@Tyler-R-Kendrick
Tyler-R-Kendrick merged commit 80fd149 into main Jul 12, 2026
3 checks passed
@Tyler-R-Kendrick
Tyler-R-Kendrick deleted the claude/harness-config-boilerplate-28r6r4 branch July 12, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants