Skip to content

WIP: docs(integrations) — reframe Claude Code and Codex as model routing - #90

Draft
SPIKESPIGEL404 wants to merge 1 commit into
mainfrom
claude/bitrouter-claude-code-codex-7af88b
Draft

WIP: docs(integrations) — reframe Claude Code and Codex as model routing#90
SPIKESPIGEL404 wants to merge 1 commit into
mainfrom
claude/bitrouter-claude-code-codex-7af88b

Conversation

@SPIKESPIGEL404

Copy link
Copy Markdown
Contributor

WIP — not ready to merge. Two blockers below, both product questions rather than writing problems.

Reframes the two integration pages from LLM-gateway wiring into model routing for long-horizon coding agents. Design rationale lives in docs/coding-agent-routing-docs-plan.md, included here so the reasoning reviews alongside the prose.

What was wrong

Both pages were: set a base URL, set a token, set a model id, read a log line — each closing with a "Learn more" pointing at the vendor's own LLM gateway doc. The routing story (policy table, loop-step fingerprints, adequacy ledger) already existed in quickstart.mdx and configuration.mdx and was never reachable from the pages where a coding-agent user actually lands.

What they are now

Both are built around bitrouter/auto. One setting hands every step of the loop to a policy; the harness's own model selection stays as the override. Hosted BitRouter is the default throughout — self-hosting is a link, not a walkthrough.

Claude Code Codex
Setting ANTHROPIC_MODEL=bitrouter/auto model = "bitrouter/auto" + model_provider
Where user ~/.claude/settings.json env block user ~/.codex/config.toml, or a profile file
Override /model (passthrough, reverts next launch) --model / /model, plus per-role models
No-file path bitrouter launch -a claude bitrouter launch --agent codex

Blockers

  1. Bare model ids on a Cloud-only account. With only ANTHROPIC_MODEL set, Claude Code still sends bare dash-form ids (claude-haiku-4-5-…) for background traffic. claude-subscription.mdx:45 documents rewriting bare Claude names on recognized Claude Code traffic, which covers the subscription path — but a Cloud-only user with no subscription attached isn't documented anywhere. If those don't resolve, the one-variable setup only works for subscribers.

  2. Dot vs dash model ids. The registry has anthropic/claude-haiku-4.5; Claude Code sends claude-haiku-4-5-…; our own docs are split 14 dash / 9 dot. This stopped being cosmetic — the rewrite path has to normalize it, and these pages tell people to paste ids into their own config files.

Deliberate calls worth a second opinion

  • One variable, not four alias slots. Pointing all of opus/sonnet/haiku/fable at bitrouter/auto would make /model a silent no-op and route Claude Code's cheap background calls through trajectory projection. ANTHROPIC_MODEL re-asserts at each launch, so an override can't become permanent.
  • The fast-path tab is launch, not a config writer. The plan specced "BitRouter writes your harness config." That doesn't exist — bitrouter launch says it touches no agent config files. The tabs are the real choice available today; a writer slots in later as a third option.
  • Codex profiles were stale everywhere. [profiles.<name>] tables were removed in Codex 0.134.0 and the top-level profile = selector is gone. Profiles are now separate ~/.codex/<name>.config.toml files with top-level keys. The old page and the plan both had this wrong.
  • Effort is stated asymmetrically on purpose. Anthropic explicitly documents that effort "shapes the rendered prompt" and breaks cached prefixes, so the Claude Code caution quotes them. OpenAI's caching docs never list reasoning effort as a segmentation factor, so the Codex caution says "can reshape" rather than asserting it. One measurement would let both become definite — see §8.1 of the plan.

Verification

  • pnpm lint:docs — OK, 45 docs
  • pnpm test — 126 passing
  • Rendered both pages locally: tabs switch, callouts style by type, no new console errors
  • Every external link and anchor fetched and confirmed, including #settings-precedence, #adjust-effort-level, #opusplan-model-setting, and the four Codex doc paths

No URLs move, so no next.config.ts redirects. lib/llms-txt.ts resynced for both pages.

Not in this PR

harnesses.mdx (the section overview) still carries the old catalog-access framing, and OpenCode / DeepSeek Harness / Pi are untouched. Plan §4.1 covers the overview rewrite — including a per-harness capability table with five cells that need verifying first.

🤖 Generated with Claude Code

Both pages were LLM-gateway wiring recipes: set a base URL, set a token,
set a model id, read a log line — each closing with a "Learn more" link to
the vendor's own LLM-gateway page. Neither connected to the routing story
that already exists in quickstart and configuration.

Both are now built around `bitrouter/auto`: one setting hands every step of
the loop to a policy, with the harness's own model selection kept as the
override. Hosted BitRouter is the default throughout; self-hosting is a
link, not a walkthrough.

- Claude Code: `ANTHROPIC_MODEL=bitrouter/auto` in the user-scope `env`
  block. Deliberately not the four alias slots — a slot table makes
  `/model` a no-op and inflates Claude Code's cheap background traffic.
- Codex: `model_provider` + `model = "bitrouter/auto"`, with per-role
  models (`review_model`, `agents.default_subagent_model`, `memories.*`)
  called out as the cache-free place to switch.
- Codex profiles corrected: `[profiles.<name>]` tables were removed in
  Codex 0.134.0; profiles are separate `~/.codex/<name>.config.toml` files.
- Every external link and anchor verified against the vendor docs.
- Adds the design plan under docs/ and resyncs lib/llms-txt.ts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@SPIKESPIGEL404 SPIKESPIGEL404 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review notes on the specific spots where I made a judgement call or am relying on something I could not verify from the docs. Six comments — the two marked BLOCKER decide whether the one-variable setup is correct as written.

"env": {
"ANTHROPIC_BASE_URL": "https://api.bitrouter.ai",
"ANTHROPIC_AUTH_TOKEN": "brk_...",
"ANTHROPIC_MODEL": "bitrouter/auto"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision: one variable, not four alias slots.

The alternative was pointing all of opus/sonnet/haiku/fable at bitrouter/auto. I rejected it because it breaks two things:

  1. /model becomes a silent no-op — the picker still lists four rows, the user picks one, nothing changes.
  2. Claude Code's background traffic (titles, small bookkeeping) goes through the Haiku slot. That is already the right routing; sending it through trajectory projection to reach the same conclusion is waste.

ANTHROPIC_MODEL also has the right precedence: it re-asserts at each launch, so an in-session /model override cannot silently become permanent.

BLOCKER tied to this: with only ANTHROPIC_MODEL set, the un-remapped slots still send bare dash-form ids (claude-haiku-4-5-…). claude-subscription.mdx:45 covers the rewrite on recognized Claude Code traffic, but a Cloud-only account with no subscription attached is undocumented. If those 404, every slot has to be remapped and this section is wrong.


Three properties matter for a long coding run:

- **It's cache-aware.** Switching models mid-session costs a prompt-cache miss on the whole prefix, so the router accounts for that when deciding whether a switch is worth making.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These three properties are written from what you told me in design discussion, not from anything currently in the docs — please confirm the wording is accurate before this ships.

Specifically: cache-aware here claims the router weighs prompt-cache cost when deciding to switch. That is the single most load-bearing claim on the page — bitrouter/auto in an interactive coding loop means trajectory changes every turn, so if the cache logic is not reliably in the path, a 150k-token session thrashes the prefix on every hop and this setup costs more than a pinned model.

This adopts your existing Claude Code session (or drives its login flow) and stores a refreshing credential. Only traffic carrying Claude Code's own agent-profile marker reaches the subscription; anything else falls through to your other configured sources — so a run can spend the plan first and continue elsewhere when a limit is reached. Multi-account labels, credential storage, and removal are on [Claude subscription](/docs/guides/claude-subscription).

<Callout type="warn">
**A fallback off the plan is a real charge.** Inside your subscription, routing is already paid for. When the policy leaves it — because a limit ran out, or because a better route lives elsewhere — those requests bill per request. Check [what actually served a run](#verify) before assuming it stayed on plan.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deliberately worded as a warning rather than a feature.

A subscriber whose run silently continues on metered models and who finds out at the invoice is a trust problem, not a support ticket. Two open items behind this:

  • Does the policy announce the hop anywhere the user will see it, or is the requests view the only signal? If it is the only signal, this callout is the entire warning a user gets.
  • model-fallback.mdx currently surfaces 402/403 and only falls through on 429. If subscription exhaustion returns 403, the "continue elsewhere when a limit is reached" claim on line 132 is false today. Worth confirming what the subscription providers actually return.

To route through BitRouter without changing what a bare `codex` does, put the selection in a [profile](https://learn.chatgpt.com/docs/config-file/config-advanced) instead. A profile is its own file at `~/.codex/<name>.config.toml`, holding **top-level keys** — not a `[profiles.<name>]` table:

```toml
# ~/.codex/bitrouter.config.toml

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the correction worth checking most carefully.

Both the old page and my own plan used [profiles.bitrouter-*] tables inside config.toml. Per Codex's advanced config docs that form was removed in 0.134.0, along with the top-level profile = "…" selector. Profiles are now separate files holding top-level keys.

I kept [model_providers.bitrouter] in ~/.codex/config.toml rather than the profile file, because provider keys are documented as user-level-only and I could not verify a profile file accepts them. If it does, this collapses to one file and the section gets shorter — worth testing against a current Codex build.

review_model = "@careful"

[agents]
default_subagent_model = "bitrouter/auto"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the one place either page tells you to switch models aggressively, and the reason is the cache argument running under the whole design: each of these roles starts a fresh context rather than continuing the main conversation, so choosing a different model costs nothing in prompt cache.

It is also the honest bit — these slots are Codex's own per-role routing. We add the model choice behind them, not the slots. Flagging in case that reads as underselling; I think it reads as credible to an audience that has read both changelogs.


<Callout type="warn">
`model_provider` / `model_providers` only take effect in the **user-level** `~/.codex/config.toml`, not a project-local `.codex/config.toml`.
**Pick an effort level per run, not per turn.** Reasoning effort can reshape the request prefix, and a mid-conversation change then costs a prompt-cache miss on the whole context — re-billing it at full input price to save a fraction of one turn's output. Set it once for the run and let plan mode carry its own value. BitRouter does not vary effort per request.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deliberate asymmetry with the Claude Code page — please do not "fix" it to match.

Anthropic explicitly documents that effort "shapes the rendered prompt" and does not preserve cached prefixes, so the Claude Code caution quotes them directly. OpenAI's prompt-caching guide never lists reasoning effort as a cache-segmentation factor and its invalidation list is prompt-structural, so this one says "can reshape" instead of asserting it.

One experiment settles both: cache a prefix, flip effort, read cache_read_input_tokens / cached_tokens. If OpenAI does not segment on effort, per-turn effort routing is viable on Codex and not on Claude Code — which is a real product asymmetry worth knowing, and would make this sentence definite in either direction. Plan §8.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant