fix(anthropic): fold the per-call runtime addendum into the preceding user message - #54
Open
zhanghanduo wants to merge 2 commits into
Open
zhanghanduo wants to merge 2 commits into
zhanghanduo wants to merge 2 commits into
Conversation
… user message The runtime addendum (system_addendum_per_call_role="user") was sent as one more user message after the tool results. Anthropic accepts that, but translating gateways do not reliably: in Forge runs r-20261001-150020-9895 and r-20261001-160244-46f2 (deepseek-flash behind llm-hub, Anthropic dialect), all six tool-pairing 400s came from apodexharness requests shaped [assistant tool_use] [user tool_result] [user addendum]; claude-code, which sends one user message per turn, had none. Each 400 made the loop drop the episode's history and answer from a recovery context. Append the addendum as a text block after the tool results instead, and do it after the cache breakpoint is placed so the breakpoint stays on the last persistent block. The prefix through that block is unchanged turn to turn, so caching behaves as before. Not reproduced after the fact: replaying the three failing requests 10 times each, as recorded and folded, returned 200 every time, so the offending backend is intermittent. The folded shape is the one every translator handles. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A reasoning-runaway retry appended its reminder as a plain user message after the transient runtime addendum. The addendum was then no longer the request's tail, so the Anthropic adapter folded nothing and the retry went out as [tool_result] [addendum] [reminder] — three consecutive user messages, the shape #54 exists to avoid. Seen in Forge run r-20261001-180502-801e: 8 of 222 apodexharness requests, all retries. The reminder is never stored in history, so mark it transient: the adapter folds both into the tool-result message and the cache breakpoint stays on the last tool_result. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With
system_addendum_per_call_role="user", each Anthropic request ends[assistant tool_use] [user tool_result…] [user addendum]. Anthropic accepts it; translating gateways do not reliably. In two Forge runs on deepseek-flash behind llm-hub (Anthropic dialect, 195 episodes), all 6 tool-pairing 400s were apodexharness requests of that shape:An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'messages.1: tool_use ids were found without tool_result blocks immediately afterclaude-code on the same model and dialect sends one user message per turn and had none. Each 400 sent ApodexHarness's loop to the clean-context forced final, and the episodes' answers were lost (3 of them on questions the other harnesses mostly solved).
#52 already folds parallel tool results into one message (unreleased since 0.12.2, so ApodexHarness's pinned image still splits them). This finishes the job for the addendum.
Change
_fold_transient_tail: after_add_prompt_cachehas placed the rolling breakpoint, append trailing transient user text to the preceding user message as text blocks. Only user→user; anything else is left alone.tool_result), so the cached prefix through that block is byte-identical turn to turn.test_anthropic_rolling_breakpoint_skips_transient_addendumnow compares through that block.test_anthropic_parallel_tool_results_share_one_user_messageupdated to the folded shape; new test for the plain-user-turn case.Also: the runaway-retry reminder is per-call
A reasoning-runaway retry (
_call.py) appended its reminder as a plain user message after the transient addendum, so the addendum was no longer the tail and nothing was folded: the retry went out as[tool_result] [addendum] [reminder]. Seen in the patched Forge rerun (r-20261001-180502-801e): 8 of 222 apodexharness requests, all retries. The reminder is never stored in history, so it is now markedtransienttoo; both fold into the tool-result message and the breakpoint stays on the lasttool_result.Verification
pytest tests: 1617 passed, 2 skipped; ruff clean. New:test_anthropic_folds_every_trailing_per_call_message,test_retry_reminder_is_per_call_so_providers_can_fold_it.anthropic.py(before the reminder fix), apodexharness on the same 24 HLE questions: tool-pairing 400s 4 → 0,stopped_by=llm_error3 → 0, timeouts 4 → 1, correct 7/24 → 11/24 (other four harnesses 10–12), billed tokens unchanged (3.53M → 3.52M).Release
ApodexHarness pins
apodex-agent-core==0.12.2; #52 and this need a release (0.12.3) and a pin bump to reach the harness image.🤖 Generated with Claude Code