WIP: docs(integrations) — reframe Claude Code and Codex as model routing - #90
WIP: docs(integrations) — reframe Claude Code and Codex as model routing#90SPIKESPIGEL404 wants to merge 1 commit into
Conversation
Both pages were LLM-gateway wiring recipes: set a base URL, set a token, set a model id, read a log line — each closing with a "Learn more" link to the vendor's own LLM-gateway page. Neither connected to the routing story that already exists in quickstart and configuration. Both are now built around `bitrouter/auto`: one setting hands every step of the loop to a policy, with the harness's own model selection kept as the override. Hosted BitRouter is the default throughout; self-hosting is a link, not a walkthrough. - Claude Code: `ANTHROPIC_MODEL=bitrouter/auto` in the user-scope `env` block. Deliberately not the four alias slots — a slot table makes `/model` a no-op and inflates Claude Code's cheap background traffic. - Codex: `model_provider` + `model = "bitrouter/auto"`, with per-role models (`review_model`, `agents.default_subagent_model`, `memories.*`) called out as the cache-free place to switch. - Codex profiles corrected: `[profiles.<name>]` tables were removed in Codex 0.134.0; profiles are separate `~/.codex/<name>.config.toml` files. - Every external link and anchor verified against the vendor docs. - Adds the design plan under docs/ and resyncs lib/llms-txt.ts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SPIKESPIGEL404
left a comment
There was a problem hiding this comment.
Review notes on the specific spots where I made a judgement call or am relying on something I could not verify from the docs. Six comments — the two marked BLOCKER decide whether the one-variable setup is correct as written.
| "env": { | ||
| "ANTHROPIC_BASE_URL": "https://api.bitrouter.ai", | ||
| "ANTHROPIC_AUTH_TOKEN": "brk_...", | ||
| "ANTHROPIC_MODEL": "bitrouter/auto" |
There was a problem hiding this comment.
Decision: one variable, not four alias slots.
The alternative was pointing all of opus/sonnet/haiku/fable at bitrouter/auto. I rejected it because it breaks two things:
/modelbecomes a silent no-op — the picker still lists four rows, the user picks one, nothing changes.- Claude Code's background traffic (titles, small bookkeeping) goes through the Haiku slot. That is already the right routing; sending it through trajectory projection to reach the same conclusion is waste.
ANTHROPIC_MODEL also has the right precedence: it re-asserts at each launch, so an in-session /model override cannot silently become permanent.
BLOCKER tied to this: with only ANTHROPIC_MODEL set, the un-remapped slots still send bare dash-form ids (claude-haiku-4-5-…). claude-subscription.mdx:45 covers the rewrite on recognized Claude Code traffic, but a Cloud-only account with no subscription attached is undocumented. If those 404, every slot has to be remapped and this section is wrong.
|
|
||
| Three properties matter for a long coding run: | ||
|
|
||
| - **It's cache-aware.** Switching models mid-session costs a prompt-cache miss on the whole prefix, so the router accounts for that when deciding whether a switch is worth making. |
There was a problem hiding this comment.
These three properties are written from what you told me in design discussion, not from anything currently in the docs — please confirm the wording is accurate before this ships.
Specifically: cache-aware here claims the router weighs prompt-cache cost when deciding to switch. That is the single most load-bearing claim on the page — bitrouter/auto in an interactive coding loop means trajectory changes every turn, so if the cache logic is not reliably in the path, a 150k-token session thrashes the prefix on every hop and this setup costs more than a pinned model.
| This adopts your existing Claude Code session (or drives its login flow) and stores a refreshing credential. Only traffic carrying Claude Code's own agent-profile marker reaches the subscription; anything else falls through to your other configured sources — so a run can spend the plan first and continue elsewhere when a limit is reached. Multi-account labels, credential storage, and removal are on [Claude subscription](/docs/guides/claude-subscription). | ||
|
|
||
| <Callout type="warn"> | ||
| **A fallback off the plan is a real charge.** Inside your subscription, routing is already paid for. When the policy leaves it — because a limit ran out, or because a better route lives elsewhere — those requests bill per request. Check [what actually served a run](#verify) before assuming it stayed on plan. |
There was a problem hiding this comment.
Deliberately worded as a warning rather than a feature.
A subscriber whose run silently continues on metered models and who finds out at the invoice is a trust problem, not a support ticket. Two open items behind this:
- Does the policy announce the hop anywhere the user will see it, or is the requests view the only signal? If it is the only signal, this callout is the entire warning a user gets.
model-fallback.mdxcurrently surfaces402/403and only falls through on429. If subscription exhaustion returns403, the "continue elsewhere when a limit is reached" claim on line 132 is false today. Worth confirming what the subscription providers actually return.
| To route through BitRouter without changing what a bare `codex` does, put the selection in a [profile](https://learn.chatgpt.com/docs/config-file/config-advanced) instead. A profile is its own file at `~/.codex/<name>.config.toml`, holding **top-level keys** — not a `[profiles.<name>]` table: | ||
|
|
||
| ```toml | ||
| # ~/.codex/bitrouter.config.toml |
There was a problem hiding this comment.
This is the correction worth checking most carefully.
Both the old page and my own plan used [profiles.bitrouter-*] tables inside config.toml. Per Codex's advanced config docs that form was removed in 0.134.0, along with the top-level profile = "…" selector. Profiles are now separate files holding top-level keys.
I kept [model_providers.bitrouter] in ~/.codex/config.toml rather than the profile file, because provider keys are documented as user-level-only and I could not verify a profile file accepts them. If it does, this collapses to one file and the section gets shorter — worth testing against a current Codex build.
| review_model = "@careful" | ||
|
|
||
| [agents] | ||
| default_subagent_model = "bitrouter/auto" |
There was a problem hiding this comment.
This is the one place either page tells you to switch models aggressively, and the reason is the cache argument running under the whole design: each of these roles starts a fresh context rather than continuing the main conversation, so choosing a different model costs nothing in prompt cache.
It is also the honest bit — these slots are Codex's own per-role routing. We add the model choice behind them, not the slots. Flagging in case that reads as underselling; I think it reads as credible to an audience that has read both changelogs.
|
|
||
| <Callout type="warn"> | ||
| `model_provider` / `model_providers` only take effect in the **user-level** `~/.codex/config.toml`, not a project-local `.codex/config.toml`. | ||
| **Pick an effort level per run, not per turn.** Reasoning effort can reshape the request prefix, and a mid-conversation change then costs a prompt-cache miss on the whole context — re-billing it at full input price to save a fraction of one turn's output. Set it once for the run and let plan mode carry its own value. BitRouter does not vary effort per request. |
There was a problem hiding this comment.
Deliberate asymmetry with the Claude Code page — please do not "fix" it to match.
Anthropic explicitly documents that effort "shapes the rendered prompt" and does not preserve cached prefixes, so the Claude Code caution quotes them directly. OpenAI's prompt-caching guide never lists reasoning effort as a cache-segmentation factor and its invalidation list is prompt-structural, so this one says "can reshape" instead of asserting it.
One experiment settles both: cache a prefix, flip effort, read cache_read_input_tokens / cached_tokens. If OpenAI does not segment on effort, per-turn effort routing is viable on Codex and not on Claude Code — which is a real product asymmetry worth knowing, and would make this sentence definite in either direction. Plan §8.1.
WIP — not ready to merge. Two blockers below, both product questions rather than writing problems.
Reframes the two integration pages from LLM-gateway wiring into model routing for long-horizon coding agents. Design rationale lives in
docs/coding-agent-routing-docs-plan.md, included here so the reasoning reviews alongside the prose.What was wrong
Both pages were: set a base URL, set a token, set a model id, read a log line — each closing with a "Learn more" pointing at the vendor's own LLM gateway doc. The routing story (policy table, loop-step fingerprints, adequacy ledger) already existed in
quickstart.mdxandconfiguration.mdxand was never reachable from the pages where a coding-agent user actually lands.What they are now
Both are built around
bitrouter/auto. One setting hands every step of the loop to a policy; the harness's own model selection stays as the override. Hosted BitRouter is the default throughout — self-hosting is a link, not a walkthrough.ANTHROPIC_MODEL=bitrouter/automodel = "bitrouter/auto"+model_provider~/.claude/settings.jsonenvblock~/.codex/config.toml, or a profile file/model(passthrough, reverts next launch)--model//model, plus per-role modelsbitrouter launch -a claudebitrouter launch --agent codexBlockers
Bare model ids on a Cloud-only account. With only
ANTHROPIC_MODELset, Claude Code still sends bare dash-form ids (claude-haiku-4-5-…) for background traffic.claude-subscription.mdx:45documents rewriting bare Claude names on recognized Claude Code traffic, which covers the subscription path — but a Cloud-only user with no subscription attached isn't documented anywhere. If those don't resolve, the one-variable setup only works for subscribers.Dot vs dash model ids. The registry has
anthropic/claude-haiku-4.5; Claude Code sendsclaude-haiku-4-5-…; our own docs are split 14 dash / 9 dot. This stopped being cosmetic — the rewrite path has to normalize it, and these pages tell people to paste ids into their own config files.Deliberate calls worth a second opinion
opus/sonnet/haiku/fableatbitrouter/autowould make/modela silent no-op and route Claude Code's cheap background calls through trajectory projection.ANTHROPIC_MODELre-asserts at each launch, so an override can't become permanent.launch, not a config writer. The plan specced "BitRouter writes your harness config." That doesn't exist —bitrouter launchsays it touches no agent config files. The tabs are the real choice available today; a writer slots in later as a third option.[profiles.<name>]tables were removed in Codex 0.134.0 and the top-levelprofile =selector is gone. Profiles are now separate~/.codex/<name>.config.tomlfiles with top-level keys. The old page and the plan both had this wrong.Verification
pnpm lint:docs— OK, 45 docspnpm test— 126 passing#settings-precedence,#adjust-effort-level,#opusplan-model-setting, and the four Codex doc pathsNo URLs move, so no
next.config.tsredirects.lib/llms-txt.tsresynced for both pages.Not in this PR
harnesses.mdx(the section overview) still carries the old catalog-access framing, and OpenCode / DeepSeek Harness / Pi are untouched. Plan §4.1 covers the overview rewrite — including a per-harness capability table with five cells that need verifying first.🤖 Generated with Claude Code