Host Claude Opus through the tool proxy

[devbadya]

Why

The on-device 0.8B model will not become ChatGPT. Visitors without a GPU (or without installing 448 MB) currently get nothing. This puts Claude Opus behind the existing Railway/tool proxy so the published Pages site can chat like a hosted product: one key on the server, no key in the bundle, tools still run in the tab.

The official Opus API is platform.claude.com (Console → API keys + Billing). That is the site to use. OpenRouter is an optional middleman if Anthropic is unset (OPENAI_BASE_URL=https://openrouter.ai/api/v1, OPENAI_MODEL=anthropic/claude-opus-5). Do not buy keys from resellers.

What changed

• POST /api/chat on the tool proxy streams Claude Opus (ANTHROPIC_API_KEY, default model claude-opus-5). OpenAI-compatible hosts remain a fallback. Set either key to mock to exercise the wiring without a provider.
• Health reports { chat: { model, provider } } when a key is set. The landing page then offers Start chatting and skips the download.
• The agent loop accepts native function calls from the hosted API (XML parsing stays for the on-device path).
• Opus 5 thinking is disabled at effort high so this proxy can rebuild tool-round messages without storing Anthropic thinking blocks (those must be echoed unmodified or the API 400s).
• When PROXY_ORIGINS is set, /api/chat requires a matching Origin so a curl from the public internet cannot spend the key.
• The header names the hosted model. The mock provider ignores skill-exemplar tool results so arithmetic still calls the calculator.

How to turn it on

1. Create an API key at platform.claude.com and add billing credit.
2. On Railway, set ANTHROPIC_API_KEY (and keep PROXY_ORIGINS pointed at the Pages origin).
3. Set the GitHub variable AGENT_API_BASE to the Railway origin, or paste that origin into Tool proxy URL.

You pay Anthropic per token for every visitor ($5 / MTok input, $25 / MTok output). Leave the key unset and inference stays in the tab, which is how the public demo still works.

Walkthrough

Mock provider (ANTHROPIC_API_KEY=mock) on pnpm dev. Landing offers Start chatting with no GPU install; header reads mock · hosted; 12 * 8 calls the calculator and answers 96.

Hosted landing with Start chatting
Calculator tool card and 12 times 8 equals 96
hosted_mock_chat_hello_and_calculator.mp4

Checks

pnpm check is green (1231 tests).

_{To show artifacts inline, enable in settings.}