Skip to content

fix(contracts): summarize the goal, not the injection preamble - #494

Open
ProtocolWarden wants to merge 1 commit into
mainfrom
fix/goal-summary-title
Open

fix(contracts): summarize the goal, not the injection preamble#494
ProtocolWarden wants to merge 1 commit into
mainfrom
fix/goal-summary-title

Conversation

@ProtocolWarden

Copy link
Copy Markdown
Owner

Problem

wrap_untrusted_goal emits GOAL_PREAMBLE before the fence, so every issue-sourced goal_text begins with "SECURITY: the text inside the <<UNTRUSTED:...".

cxrp_mapper then sliced that raw string for two short fields:

title=oc.goal_text[:80],    # to_cxrp_task_proposal
scope=oc.goal_text[:120],   # to_cxrp_execution_request

So every issue-sourced task was titled and scoped with the preamble's opening words instead of its actual request.

Visible live on two open PRs, both titled SECURITY: the text inside the <<UNTRUSTED:...>> … <</UNTRUSTED:...>> fen:

PR Real goal
#478 Fix edge_cases to forward the sample list, not the count dict
#483 Add regression test suite that execs the live STEP 3 snippet against the OUTPUT

Cosmetic in effect, corrosive in practice: routine autonomous PRs read as security events, and the board becomes unscannable. Both call sites are the same bug — fixing only the title would have left scope broken.

Change

The fix is not a regex in the mapper. injection.py owns the fence format, so the reader lives there:

  • unfence_goal() — extracts the fenced payload. The closing marker is matched by backreference on the nonce, mirroring fence's guarantee that a forged close marker with a guessed nonce does not terminate the span. Falls back to the input unchanged when unfenced.
  • goal_summary() — unfence → collapse to one line → sanitize_for_comment → bound to max_len.

The mapper just calls goal_summary. The 80/120 bounds are preserved (the CxRP schemas impose no length constraint on these fields — verified).

Two deliberate decisions

objective still carries the FULL wrapped text. The preamble and fence must reach the executor intact; only the short human/telemetry-facing fields are summarized. test_task_proposal_objective_keeps_the_full_fenced_goal pins that distinction so a future "cleanup" cannot strip the fence from the executor's input.

This moves attacker-influenced text into GitHub PR titles, which the old (accidental) behaviour did not do. goal_summary therefore routes through sanitize_for_comment to defang @mentions — a bare @handle in a PR title pings a real person — and strip zero-width/bidi characters. Single-line collapse matters for the same reason: a newline breaks a title.

Verification

Verified by mutation, not just green tests. Reverting both call sites to the raw slices makes the new pins fail, reproducing the exact observed string:

E  AssertionError: assert 'SECURITY: th...from an exter' == 'Add a regres...TEP 3 snippet'
FAILED test_task_proposal_title_summarizes_request_not_preamble
FAILED test_execution_request_scope_summarizes_request_not_preamble

Restored, all pass.

  • 18 new tests (13 in test_injection.py, 5 in test_cxrp_mapper.py)
  • Full unit suite: 8548 passed, 0 failed (8530 → 8548, exactly the new tests)
  • No pre-existing test asserts on CxRP title/scope, so blast radius is limited to the new pins
  • ruff check + ruff format clean; Custodian pre-push gate clean

Note

This does not change what the executor receives, only what humans and telemetry see. #478 and #483 keep their current titles — GitHub titles are set at creation; this fixes titles generated from here on.

🤖 Generated with Claude Code

wrap_untrusted_goal emits GOAL_PREAMBLE before the fence, so every issue-sourced
goal_text starts with "SECURITY: the text inside the <<UNTRUSTED:...". cxrp_mapper
sliced that raw string for two short fields — title=goal_text[:80] and
scope=goal_text[:120] — so every issue-sourced task was titled and scoped with
the preamble instead of its actual request.

Visible live on PRs #478 and #483, whose titles both read "SECURITY: the text
inside the <<UNTRUSTED:...>> ... fen" while their real goals were "Fix
edge_cases to forward the sample list, not the count dict" and "Add regression
test suite that execs the live STEP 3 snippet against the OUTPUT". Both call
sites were the same bug; fixing only the title would have left scope broken.

injection.py owns the fence format, so the reader lives there rather than as a
regex in the mapper: unfence_goal() extracts the payload (backreferenced nonce,
so a forged close marker with a guessed nonce does not terminate the span) and
goal_summary() unfences, collapses to one line, defangs, and bounds.

Two deliberate decisions:

- objective still carries the FULL wrapped text. The preamble and fence must
  reach the executor intact; only the short human/telemetry-facing fields are
  summarized. A test pins that distinction.
- This MOVES attacker-influenced text into GitHub PR titles, which the old
  accidental behavior did not. goal_summary therefore routes through
  sanitize_for_comment to defang @mentions and strip zero-width/bidi chars, and
  collapses newlines, which would otherwise break a title.

Verified by mutation: reverting both call sites to the raw slices makes the new
pins fail, reproducing the observed string (scope == 'SECURITY: th...from an
exter'); restored, all pass. 44 tests across test_injection.py and
test_cxrp_mapper.py. No pre-existing test asserts on CxRP title/scope. ruff
check and format clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ProtocolWarden
ProtocolWarden force-pushed the fix/goal-summary-title branch from f065f30 to 44916ee Compare August 4, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant