Skip to content

fix(anthropic): fold the per-call runtime addendum into the preceding user message - #54

Open
zhanghanduo wants to merge 2 commits into
mainfrom
fix/anthropic-fold-transient-addendum
Open

zhanghanduo wants to merge 2 commits into
mainfrom
fix/anthropic-fold-transient-addendum

Conversation

@zhanghanduo

@zhanghanduo zhanghanduo commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

With system_addendum_per_call_role="user", each Anthropic request ends [assistant tool_use] [user tool_result…] [user addendum]. Anthropic accepts it; translating gateways do not reliably. In two Forge runs on deepseek-flash behind llm-hub (Anthropic dialect, 195 episodes), all 6 tool-pairing 400s were apodexharness requests of that shape:

  • An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'
  • messages.1: tool_use ids were found without tool_result blocks immediately after

claude-code on the same model and dialect sends one user message per turn and had none. Each 400 sent ApodexHarness's loop to the clean-context forced final, and the episodes' answers were lost (3 of them on questions the other harnesses mostly solved).

#52 already folds parallel tool results into one message (unreleased since 0.12.2, so ApodexHarness's pinned image still splits them). This finishes the job for the addendum.

Change

_fold_transient_tail: after _add_prompt_cache has placed the rolling breakpoint, append trailing transient user text to the preceding user message as text blocks. Only user→user; anything else is left alone.

  • The breakpoint stays on the last persistent block (the last tool_result), so the cached prefix through that block is byte-identical turn to turn. test_anthropic_rolling_breakpoint_skips_transient_addendum now compares through that block.
  • test_anthropic_parallel_tool_results_share_one_user_message updated to the folded shape; new test for the plain-user-turn case.

Also: the runaway-retry reminder is per-call

A reasoning-runaway retry (_call.py) appended its reminder as a plain user message after the transient addendum, so the addendum was no longer the tail and nothing was folded: the retry went out as [tool_result] [addendum] [reminder]. Seen in the patched Forge rerun (r-20261001-180502-801e): 8 of 222 apodexharness requests, all retries. The reminder is never stored in history, so it is now marked transient too; both fold into the tool-result message and the breakpoint stays on the last tool_result.

Verification

  • pytest tests: 1617 passed, 2 skipped; ruff clean. New: test_anthropic_folds_every_trailing_per_call_message, test_retry_reminder_is_per_call_so_providers_can_fold_it.
  • Forge rerun with this branch's anthropic.py (before the reminder fix), apodexharness on the same 24 HLE questions: tool-pairing 400s 4 → 0, stopped_by=llm_error 3 → 0, timeouts 4 → 1, correct 7/24 → 11/24 (other four harnesses 10–12), billed tokens unchanged (3.53M → 3.52M).
  • Not reproduced after the fact: replaying the three failing requests 10× each, as recorded and folded, returned 200 every time (the offending backend is intermittent). So this is the canonical shape, not a measured fix.

Release

ApodexHarness pins apodex-agent-core==0.12.2; #52 and this need a release (0.12.3) and a pin bump to reach the harness image.

🤖 Generated with Claude Code

zhanghanduo and others added 2 commits October 1, 2026 17:59
… user message

The runtime addendum (system_addendum_per_call_role="user") was sent as one
more user message after the tool results. Anthropic accepts that, but
translating gateways do not reliably: in Forge runs r-20261001-150020-9895
and r-20261001-160244-46f2 (deepseek-flash behind llm-hub, Anthropic
dialect), all six tool-pairing 400s came from apodexharness requests shaped
[assistant tool_use] [user tool_result] [user addendum]; claude-code, which
sends one user message per turn, had none. Each 400 made the loop drop the
episode's history and answer from a recovery context.

Append the addendum as a text block after the tool results instead, and
do it after the cache breakpoint is placed so the breakpoint stays on the
last persistent block. The prefix through that block is unchanged turn to
turn, so caching behaves as before.

Not reproduced after the fact: replaying the three failing requests 10
times each, as recorded and folded, returned 200 every time, so the
offending backend is intermittent. The folded shape is the one every
translator handles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A reasoning-runaway retry appended its reminder as a plain user message
after the transient runtime addendum. The addendum was then no longer the
request's tail, so the Anthropic adapter folded nothing and the retry went
out as [tool_result] [addendum] [reminder] — three consecutive user
messages, the shape #54 exists to avoid. Seen in Forge run
r-20261001-180502-801e: 8 of 222 apodexharness requests, all retries.

The reminder is never stored in history, so mark it transient: the adapter
folds both into the tool-result message and the cache breakpoint stays on
the last tool_result.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant