Skip to content

feat(daemon): add opt-in local OpenAI/Anthropic API proxy - #30

Merged
jazelly merged 1 commit into
devfrom
feat/local-api-proxy
Aug 10, 2026
Merged

feat(daemon): add opt-in local OpenAI/Anthropic API proxy#30
jazelly merged 1 commit into
devfrom
feat/local-api-proxy

Conversation

@termi-tao

Copy link
Copy Markdown
Collaborator

What

Lets an existing project that already speaks the OpenAI or Anthropic wire format route its single-shot Q&A traffic through a visible provider session instead of a paid API, by pointing base_url at the daemon.

New routes on the existing daemon HTTP server, behind the existing control bearer token:

Method Path
POST /v1/openai/chat/completions
GET /v1/openai/models
POST /v1/anthropic/messages

Both dialects normalize into one job submission that reuses createManagedPlaywrightJobRequestprompt.input / prompt.submit / response.read, then polls to a terminal state.

This is a task-level bridge, not a drop-in API replacement. It trades throughput, latency, streaming, and tool use for cost.

Design decisions

Disabled by default. It accepts local API traffic, so enabling it without an explicit request would widen what the daemon answers. tokenless api-proxy <status|enable|disable> plus an interactive setup step turn it on.

Explicit tokenless/<provider> model naming. An unmapped model such as gpt-4o is rejected rather than redirected, so it stays visible which account and which subscription answered.

Two conversation modes, chosen at setup or with --conversation-mode:

  • new-conversation flattens the transcript into one prompt and starts a fresh provider conversation per request. Stateless.
  • continue-conversation fingerprints every message except the final user turn and reuses one provider conversation for that thread. A caller that edits or truncates history starts a new conversation instead of appending to a transcript the provider no longer shares.

Fails closed on what visible pages cannot do. tools, tool_choice, functions, function_call, and response_format return 400 — returning prose where the caller expects a tool call would be worse than an error.

stream: true replays each dialect's documented event sequence as one terminal chunk. A visible reply is only readable once it has finished rendering, so there is no partial text to forward; refusing stream would break otherwise compatible clients for no benefit.

Reported usage is always zero. Tokenless does not meter provider tokens — the reply is billed by the caller's own web subscription, so a fabricated count would be worse than none.

Also included

Linux x64 support in the managed browser catalog, which was needed to exercise any of this on WSL:

  • cloak:linux-x64 (CloakBrowser 146.0.7680.177, sha256 checked against the release SHA256SUMS) and managed-chromium:linux-x64 (Chrome for Testing 146.0.7680.165). The second one is required because auto is the default selection and would otherwise throw No managed browser catalog entry on Linux.
  • systemBrowserExecutable previously fell through to the Windows LOCALAPPDATA / PROGRAMFILES branch on Linux. It now discovers /opt/google/chrome/chrome and friends.
  • Extraction used tar for every archive. GNU tar cannot read zip, so managed-chromium and auto failed on Linux with tar: This does not look like a tar archive. Zip now goes through unzip, and the archive path-traversal check switches from tar -tf to unzip -Z1 so the security check is not bypassed. Note: this makes unzip a hard dependency on Linux.

Verification

Real boundaries only, no mocks:

  • New test/api-proxy-local-http.integration.test.mjs — 8 tests, all passing, against a real daemon over real HTTP with a real filesystem.
  • Both managed runtimes installed end to end on Linux through the real download → verify → extract → version → smoke-launch → install path, with checksumVerified=true.
  • Documented contract cross-checked against the live endpoints: 401/403 codes, the model list and its ordering, developer role accepted, non-text content parts rejected, each rejected field's message, sampling fields ignored rather than rejected, the 256-message cap, and Anthropic's role restriction. 9/9 held.
  • npm run lint and npm run api:check clean.

Pre-existing failures, unchanged by this branch: test/package-contract.test.mjs has 7 and test/local-web-control-plane.test.mjs has 1. Verified identical with the branch changes stashed; they stem from packages/extension existing and global-install behavior.

Docs

  • docs/api-proxy-integration.md and its zh-CN partner — the integration contract, written for an implementer.
  • tokenless api-proxy sections in both COMMANDS references.
  • Both docs/README indexes updated.
  • .gitignore gains two allowlist entries: docs/* is allowlist-ignored, so the new docs would not have been committed while the tracked indexes linked to them.

Known gap, not addressed here

Every non-auth failure returns 400 / invalid_input regardless of cause, so a client cannot tell a malformed request from a transient provider blocker and can only string-match message. The integration doc says so explicitly and tells clients not to auto-retry. Mapping these onto 401/403/409/429/504 with a stable machine-readable code is worth a follow-up — a real provider_rate_limited false positive was already hit in testing.

🤖 Generated with Claude Code

@jazelly
jazelly self-requested a review August 10, 2026 13:04
Existing projects that already speak the OpenAI or Anthropic wire format
can now route their single-shot Q&A traffic through a visible provider
session instead of a paid API, by pointing base_url at the daemon.

New daemon routes, behind the existing control bearer token:

  POST /v1/openai/chat/completions
  GET  /v1/openai/models
  POST /v1/anthropic/messages

The proxy ships disabled. `tokenless api-proxy <status|enable|disable>`
and an interactive setup step turn it on and pick how proxied turns map
onto provider conversations: `new-conversation` flattens the transcript
and starts a fresh conversation per request, while
`continue-conversation` fingerprints all but the final user turn and
reuses one provider conversation for that thread.

`model` must name the provider explicitly as `tokenless/<provider>`. An
unmapped model is rejected rather than redirected, so it stays visible
which account and which subscription answered. Tool, function, and
structured-output fields fail closed because visible pages expose no
equivalent control. `stream: true` replays each dialect's documented
event sequence as one terminal chunk, since a visible reply is only
readable once it has finished rendering. Reported usage is always zero:
Tokenless does not meter provider tokens.

Also adds Linux x64 to the managed browser catalog with pinned
CloakBrowser 146 and Chrome for Testing 146, discovers system browsers
from their Linux install paths instead of falling through to the Windows
branch, and extracts zip artifacts with unzip where GNU tar cannot read
them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@termi-tao
termi-tao force-pushed the feat/local-api-proxy branch from 7b7bb5b to cf23603 Compare August 10, 2026 13:15
@termi-tao

Copy link
Copy Markdown
Collaborator Author

Rebased onto current dev

The first push was branched from a stale local dev that was 23 commits behind origin/dev — my mistake, I never fetched before branching. Now rebased onto abf8626; mergeable: MERGEABLE.

Resolving it changed the scope, because upstream had independently done part of this work:

Dropped as superseded — upstream already added cloak:linux-x64 (plus darwin-x64 and linux-arm64) with the identical sha256, and already widened BrowserRuntimePlatform and the platform gate to five platforms. My versions of those are gone; a duplicate cloak:linux-x64 key that the auto-merge produced was removed.

Still needed, and kept — upstream's Linux support was incomplete:

  • managed-chromium:linux-x64 is absent upstream, so auto (the default selection, manager.ts:117) still throws No managed browser catalog entry on Linux.
  • systemBrowserExecutable still falls through to the Windows LOCALAPPDATA / PROGRAMFILES branch on Linux. Added a Linux branch, now including Brave since upstream added it to SYSTEM_BROWSER_IDS.
  • Extraction still uses tar for every archive, and GNU tar cannot read zip. Chrome for Testing ships zip, so the entry above needs the unzip path.

Reworked against upstream refactors:

  • TokenlessConfig.providerWhitelist and profilePreferences are now config.profiles, so the proxy resolves the managed profile and reads enabledProviders from it. The error text changed from "not enabled for this installation" to "not enabled for the managed profile", and api-proxy status now reports the resolved profile and accepts --profile.
  • Localization moved from an English→Chinese Map to the keyed i18n/catalog.ts system. The four setup strings are now catalog keys rendered through t(), and the two new error codes were added to ERROR_SUMMARIES_ZH.
  • Validation order was corrected while adapting: an unregistered provider is a request-level error, so it is now rejected before the profile is resolved. Otherwise a missing profile returned 409 where the caller's own bad model deserved 400.

Excluded from the commit: package-lock.json. npm install (needed to pick up upstream's new impers dependency) stripped libc fields from optional platform packages — pure npm-version churn, no dependency change of mine. Restored to origin/dev's version.

Re-verified on the new base: lint, api:check, and build clean; the 8 api-proxy integration tests pass; test/package-contract.test.mjs shows the same 7 pre-existing failures (numbering shifted only because upstream added tests).

Unrelated papercut noticed: upstream's build script writes a tokenless-beta launcher at the repo root, and it is not in .gitignore, so it shows up untracked after every npm run build. Not touched here.

@jazelly
jazelly merged commit ab7ba9d into dev Aug 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants