Control AI coding spend. Require proof before merge.
Runcap is a local-first control layer for AI coding agents. It can cap routed AI spend and use a pinned GitHub Action to replay base-pinned verification before an AI-generated PR earns merge eligibility.
Current public position:
Runcap controls AI coding missions.
It caps routed AI spend,
constrains what an agent may change,
and requires proof before an AI-generated pull request earns merge eligibility.
What Runcap sees:
- Spend controls apply only to requests routed through Runcap.
- The CI Proof Gate independently verifies code changes; it does not claim to independently meter all model usage.
- Hosted sync, team pools, organization reporting, and paid plans are not products available for purchase today.
See current product status for the exact public boundary.
An agent passing CI is not enough. Runcap verifies whether the evidence of success was altered during the mission.
Estimate the run → Cap the spend → Verify the outcome
If Runcap caps a run for you or compresses a call, please star the repo - it is the one signal that tells me to keep building it in the open.
AI can propose a change. Runcap makes it earn merge eligibility.
runcap ci --mode adjudicate is a required PR check that does not trust the agent or its receipt. It recomputes the merge decision in a clean CI job from the pull request's base commit:
AI-generated PR
→ Runcap action pinned to an immutable release commit
→ policy / verifier / dependencies read from the PR base commit
→ clean CI replay
→ PASS / BLOCKED / HUMAN_APPROVAL_REQUIRED
PASS- the base verifier failed, the replay passed, and the change was allowed text-only edits inside scope.BLOCKED- a scope violation, an unsafe diff type (delete / rename / binary / symlink / submodule / mode change), an unresolved base/head identity, or a failed replay.HUMAN_APPROVAL_REQUIRED- the change touches the policy, a workflow, a verifier file, a dependency manifest/lockfile, or a protected path. Runcap does not auto-approve changes to its own rules or evidence; a human CODEOWNER must approve.
The verdict is a CI-attested replay under a documented hardened GitHub profile. It is not "unspoofable," not "fully independent," and it is not independent budget enforcement - its integrity rests on the required GitHub setup being in place. The agent's receipt never decides the verdict: the required gate does not read it.
See the trust model for exactly what v0.6 proves and what it does not, and Install in a consumer repo to wire it up.
Agents loop on the same error, rewrite plans, and re-read files they just edited - every loop is tokens you pay for. Multi-agent coding runs burn roughly 15x more tokens than a single chat (Anthropic engineering). They hand you a confident summary while the task is not actually done, and you find out what it cost when the invoice - or the subscription limit - arrives.
Observability tools (Langfuse, Helicone, LangSmith, AgentOps) measure the past, and some run evals on outputs. Gateways (LiteLLM, Portkey, OpenRouter) route the present. What they don't do is enforce a mission policy during the run - a hard spend cap, allowed scope, protected verification evidence - then issue evidence about whether the agent's own success check can be trusted. Runcap does the things the rear-view mirror can't:
estimate before build → cap during run → compress every call → rescue when stuck → verify the outcome
It also quietly trims waste on the way through: on a real OpenAI call, one edited-file re-read dropped from 1,186 to 737 prompt tokens (37.9% saved) with the model still answering correctly about the changed line - see the details below.
Runcap does not promise an exact cost oracle. Agent trajectories are stochastic - nobody, including the model labs, can predict the exact token count of a run. So Runcap gives you a range plus a hard cap:
"This build is roughly $3-7. Cap it at $10." - then it kills the run the second it hits the ceiling.
The range is the headline. The hard cap is the product.
Runcap is a developer tool. It works by running a local gateway that your agent's API calls pass through, so it can price and cap them before they reach the paid provider. That means you need three things already in place:
- Your own provider API key (OpenAI or Anthropic). Runcap does not sell or supply model access.
- Your own agent - Claude Code, Codex, or any script that calls the OpenAI/Anthropic API.
- Comfort running a CLI and a local process on your machine.
If you have those, Runcap caps your spend in one command. If you are looking for a no-account web app that runs the AI for you, this is not that - it is a circuit breaker for a setup you already own.
No API key required.
git clone https://github.com/kirder24-code/ai-agent-manager.git
cd ai-agent-manager
npm run setup
npm run demo1. Catch a too-broad request before it spends anything:
$ runcap preflight -- claude "build the full mobile app with auth payments and production deploy"
Preflight: claude build the full mobile app with auth payments and production deploy
Scope risk: high
Fuel: 24% (medium confidence)
Recommendation: Do not launch as one broad mission. Split into one vertical slice with a verification command.
2. Wrap a run - and get a rescue prompt the moment it gets stuck:
$ runcap run --label demo -- npm run build
Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@/components' ...
Runcap mission: 20260601T221531-demo-ff42c0a
Status: stuck (medium confidence)
Exit code: 1
Changed files: 0
Parsed errors: 1
Primary recommendation: Resolve missing import before continuing feature work
The rescue report hands back a copyable prompt:
Do not continue broad implementation. Diagnose this missing module first:
Cannot find package '@/components'. Check package.json, tsconfig paths, and
the latest git diff. Make the smallest change that resolves the import,
then run the failing command again.
npm install -g runcap # exposes `runcap` (and `aim` as a legacy alias)Or run from source with node ./bin/runcap.mjs <command>.
runcap plan --fuel 24 -- "build a small auth feature and verify it" # range + recommended cap, before you spend
runcap preflight -- claude "build a full SaaS app" # is this prompt too broad?
runcap run --label fix -- claude "fix one failing check. stop if blocked." # wrap any agent/command
runcap outcome run --task "..." --verify "pnpm test" -- claude "fix it" # cost of a VERIFIED result, not tokens
runcap report # human-readable rescue report
runcap export # evidence JSON with truth labels
runcap dashboard # local cockpit at :8791
runcap gateway # cost-tracking proxy with hard budget cap
runcap fuel set 24 # calibrate a %-only subscriptionPoint any OpenAI- or Anthropic-compatible tool at the local gateway. It records real token usage, prices it from a sourced table, and blocks calls the moment your daily ceiling is hit.
# OpenAI-compatible agents
OPENAI_API_KEY=sk-... AIM_DAILY_BUDGET_USD=5 runcap gateway
# then: OPENAI_BASE_URL=http://127.0.0.1:8792/v1
# Anthropic-native (Claude Code, /v1/messages)
ANTHROPIC_API_KEY=sk-ant-... AIM_DAILY_BUDGET_USD=5 runcap gateway
# then: ANTHROPIC_BASE_URL=http://127.0.0.1:8792/v1
# DeepSeek (OpenAI-compatible, much cheaper - same one command)
OPENAI_API_KEY=sk-... AIM_UPSTREAM_BASE_URL=https://api.deepseek.com AIM_DAILY_BUDGET_USD=5 runcap gateway
# then point your agent at: OPENAI_BASE_URL=http://127.0.0.1:8792/v1 (model: deepseek-chat)When spend crosses the ceiling, the next call returns 429 budget_guard instead of money leaving your account. Try it with no key: runcap gateway --mock.
Every request that passes through the gateway is compressed before it's forwarded. Three layers, all lossless by construction - your prose instructions and code semantics are never altered, only machine "garbage" is trimmed:
- Per-field trim - embedded JSON re-serialized compactly, long log/stack-trace dumps collapsed to head + tail, trailing whitespace squeezed.
- Identical-block dedup - when the exact same file dump or tool_result ships again in the same request, the repeat is replaced with a deterministic stub.
- Delta-encoding of near-duplicates. When the agent reads a file, edits one line, and re-reads it, the block is similar but not identical, so plain dedup saves nothing. Runcap sends a readable line-diff against the version the model already saw, and the model reconstructs the current file from it. On a real OpenAI call, an edited-file re-read dropped from 1186 to 737 prompt tokens - 37.9% saved, with the model still answering correctly about the changed line. Proof and reproduction steps: docs/delta-encoding-evidence.md.
It's pure Node with zero native or ML dependencies (the only runtime dependency is js-yaml, pure JS), so it installs everywhere without the build pain heavier compressors have.
The dashboard shows the result as one number: "You saved $X · N tokens compressed · would have spent $Y." Disable it with AIM_COMPRESS=off if you ever want raw passthrough.
The hard case in stuck-detection is the agent that keeps producing output but is really circling the same failure, just reworded each time. Plain hashing misses it because the prompt is similar but never byte-identical between loops. Because the gateway sees every request, Runcap compares each request's conversation shape against the recent run with the same line-similarity primitive the delta-encoder uses: when several prompts in a row are near-identical (default: 3 prompts at 92%+ similarity) while the conversation never moves forward, it flags loop.looping on the event, surfaces a warning in runcap status, and fires an alert.
This is a calculated signal, not a proven dollar-saving: it tells you "the agent has sent 3 near-identical prompts in a row with no progress" so you can step in before the loop burns more budget. Tune or disable it with AIM_LOOP_DETECT=off. (Today's detectStuck post-run score is outcome-based: exit code, parsed errors, and zero-diff. The loop signal adds the missing in-flight behavioral signal on top of it.)
Tokens are the wrong unit. You don't buy tokens, you buy a result that passes a check. So Runcap measures the only number that tracks what you actually paid for:
Verified Outcome Cost = total run cost / tasks that passed verification.
Wrap your agent and hand it a verification command. Its exit code is the oracle:
runcap outcome run \
--task "Fix the failing test" \
--verify "pnpm test && pnpm build" \
-- claude "fix one failing check, then stop"
runcap outcome show # re-print the latest receiptThe run produces an outcome receipt where every field carries a truth label:
Outcome: VERIFIED
Actual cost: $0.000677 (2 priced LLM calls, calculated_from_provider_usage_and_sourced_price_table)
Verified Outcome Cost: $0.000677 (money that bought a verified result)
If verification fails, the receipt is honest about it instead of pretending the spend delivered something:
Outcome: UNVERIFIED
Verified Outcome Cost: N/A (verification did not pass)
Money spent without verified delivery: $0.001012
That second case is the whole point: an agent that talks but never fixes the bug can cost more than one that does, while a token dashboard calls it "cheaper." Try both, offline, no API key:
runcap outcome run --task "Fix sum() so it adds" --verify "node examples/outcome-demo/verify.mjs" --mock -- node examples/outcome-demo/agent-fixes.mjs # VERIFIED
runcap outcome run --task "Fix sum() so it adds" --verify "node examples/outcome-demo/verify.mjs" --mock -- node examples/outcome-demo/agent-spins.mjs # UNVERIFIEDReceipts are written to .runcap/outcomes/<id>/receipt.json. Run the same task across several agents and you get the Agent Economics Index - the same board, priced by verified delivery instead of tokens.
Same task - fix a broken sum() so the test passes - across different models and two agent behaviors. Every number below is measured from the gateway ledger, not estimated:
| # | Model | Agent | LLM calls | Actual cost | Outcome | Verified Outcome Cost | Money, nothing delivered |
|---|---|---|---|---|---|---|---|
| 1 | gpt-4o | writes fix | 2 | $0.000677 | VERIFIED | $0.000677 | $0 |
| 2 | gpt-4o | spins, no fix | 3 | $0.001012 | UNVERIFIED | N/A | $0.001012 |
| 3 | gpt-5.4 | writes fix | 2 | $0.000957 | VERIFIED | $0.000957 | $0 |
| 4 | deepseek-chat | writes fix | 2 | $0.000022 | VERIFIED | $0.000022 | $0 |
| 5 | deepseek-chat | spins, no fix | 3 | $0.000033 | UNVERIFIED | N/A | $0.000033 |
| 6 | claude-sonnet-4 | writes fix | 2 | $0.000981 | VERIFIED | $0.000981 | $0 |
Two facts a token dashboard can't show you: on gpt-4o the run that delivered nothing (row 2) cost more than the run that delivered (row 1); and the cheapest verified result (deepseek-chat, $0.000022) bought the same passing test as gpt-5.4 at ~43x the price. Reproduce any row with OUTCOME_DEMO_MODEL=<model> in front of the command above. (One run per row in v0.1 - the point is the unit, not a vendor ranking; a ranking needs N>=5 runs/agent and a pass-rate column.)
A green test only means something if the agent passed it fairly. An agent under pressure can turn a check green without doing the work: delete the failing test, rewrite the assertion, repoint the npm test script at true, disable TypeScript strict mode, mock the real API, or hardcode the expected answer. Exit code 0 - and a plain Verified Outcome Cost calls it delivered. So does every token dashboard.
--guard verifies the verifier. Before the agent runs, Runcap freezes a Task Contract: the baseline git commit, a SHA-256 of every file the verify command names, a snapshot of package.json scripts, and a check that the task actually fails today (a pass on an already-green tree proves nothing). After the run it re-checks all of it, and re-runs the verify command from the baseline commit in a throwaway git worktree with only the agent's changed files copied in - so a green that depended on uncommitted local junk dies in the clean room.
The result is a trust grade, not a binary:
| Status | Meaning |
|---|---|
VERIFIED_STRONG |
Passed, verifier untouched, the task really failed before, and the pass survives a clean checkout. |
VERIFIED_WEAK |
Passed and verifier untouched, but a strong condition was missed (e.g. baseline failure not reproduced). |
UNVERIFIED |
Verification did not pass. |
VERIFIER_COMPROMISED |
Passed, but the verifier itself was modified during the run. The green light cannot be trusted. |
runcap outcome run --guard \
--task "Fix the failing test" \
--verify "pnpm test" \
--allow src/ \
-- claude "fix one failing check, then stop"--protect <path> marks extra paths the agent must not touch (tests, config, and package.json are protected by default); --allow <path> declares the only paths a legitimate fix should change, so out-of-scope edits drop the grade. The receipt gains a verificationIntegrity block listing every check, every truth label, and exactly which file was tampered with if the grade is VERIFIER_COMPROMISED.
One honesty note that rides on every receipt: Verified Outcome Cost is the LLM spend that bought the result - it does not include subscriptions, CI minutes, sandbox compute, or human review time. For real agent economics you want Expected Verified Outcome Cost = total spend across N attempts / strongly-verified outcomes, which needs N>=5 runs. That's the v0.2 unit; v0.1 measures the one cost the gateway can observe honestly.
Everything above lives in one developer's terminal. A platform, VP-Eng, or FinOps owner can't act on it: there's no way to declare the rules of a mission once, in the repo, and no way to fail a pull request when an agent breaches budget or tampers with the evidence of its own success.
A mission policy closes that gap. You declare the rules once in .runcap/mission.yaml, enforce them during the run, and grade the result into a PASS / BLOCKED verdict a GitHub Action turns into a red/green check on the PR.
# .runcap/mission.yaml
version: v1
identity:
project: checkout
team: payments
mission:
name: Fix the failing checkout test
task_class: bugfix
budget:
mission_hard_limit_usd: 10 # required - per-mission hard cap (the gateway enforces it live)
max_llm_calls: 12 # optional - BLOCK if exceeded
max_runtime_minutes: 30 # optional - BLOCK if exceeded
verification:
command: "pnpm test && pnpm build" # required - the oracle (exit 0 = delivered)
guard: strict # strict (default) freezes + re-checks the verifier
protect: ["tests/**", "package.json"] # paths the agent must not touch
allow: ["src/checkout/**"] # the only paths a legit fix should changeValidate it, then run the agent under it:
runcap policy validate # is .runcap/mission.yaml well-formed?
runcap mission run -- claude "fix the failing checkout test, then stop"mission run enforces the per-mission hard cap through the gateway, always runs the verification guard, and writes an outcome receipt that now carries a policy block - the org attribution, the limits, the SHA-256 of the exact policy text that graded the run, and the verdict. It exits 1 on BLOCKED, so it fails CI on its own. The mission is BLOCKED when any of these is true:
- the verifier was tampered with (
VERIFIER_COMPROMISED); - verification did not pass (
UNVERIFIED); - a change landed outside the declared
allowscope; - spend exceeded
mission_hard_limit_usd, or the gateway's budget guard tripped mid-run; max_llm_callsormax_runtime_minuteswas exceeded.
There are two ways the policy verdict reaches a PR, and they trust different things:
runcap mission run(local / same-job) grades the run it just executed and re-checks it against the committed policy text. Useful, but the receipt it produces is agent-side evidence.runcap ci --mode adjudicate(the required PR check) trusts none of that. It recomputes the verdict in a clean CI job from the PR's base commit and never reads the agent receipt. This is the gate that decides merge eligibility.
Make the adjudication a required red/green PR check in your own repo:
- Add
.runcap/mission.yaml(the policy - see the example above). - Copy
examples/runcap-adjudicate.ymlinto.github/workflows/. - The template is pinned to Runcap v0.6.0. When intentionally upgrading, replace the SHA with the full 40-character target commit SHA shown for the GitHub Release you choose. For local verification in a clone of the Runcap repository, peel an annotated tag to its commit with
git rev-parse "vX.Y.Z^{}". Do not use an annotated tag object SHA - for the Proof Gate, pin the commit SHA that the release tag resolves to. Never use@v1or another floating tag for the Proof Gate. Use a full immutable commit SHA. - Configure the hardened GitHub branch profile (protected branch, required check, up-to-date-before-merge, dismiss stale approvals, CODEOWNERS for workflow/policy/verifier/dependency/protected paths, no bypass for ordinary authors) - the full list is in the trust model.
- Make
Runcap adjudicatea required status check.
The template is pinned to the Runcap v0.6.0 release commit. This is deliberate: the judge must be an immutable release commit that lives outside the candidate PR, so a malicious PR cannot rewrite its own judge.
A reviewer sees one of two things:
Mission verdict: PASS
project checkout / team payments
Mission cost $0.0007 / $10.00
Policy hash: c857d10c…
Mission verdict: BLOCKED
Blocked because:
- VERIFIER_COMPROMISED: the agent changed protected verification evidence (verifier_file_unchanged:app/verify.mjs).
Because the verdict is recomputed from the committed policy and the receipt records the policy hash, a reviewer can confirm which rules graded the run - the verdict is only as trustworthy as the policy hash it carries.
See a public consumer-repository demo with three live pull requests: an in-scope source fix that passes, an out-of-scope change that is blocked, and a verifier edit that requires human approval.
- Demo repo: kirder24-code/runcap-proof-gate-demo
- PR #1 - in-scope source fix -
PASS - PR #2 - out-of-scope change -
BLOCKED - PR #3 - verifier edit -
HUMAN_APPROVAL_REQUIRED
Each verdict was produced by the pinned Runcap action running in GitHub Actions on a real pull request.
Watch a 70-second screen recording made from the public consumer demo and its real GitHub Actions runs. It shows one problem and three verdicts:
An AI-generated PR can make CI green by changing the test that proves it succeeded.
- a scoped source fix →
PASS - a correct fix plus an unrelated file →
BLOCKED - a verifier edit →
HUMAN_APPROVAL_REQUIRED
This is a CI-attested replay under a documented hardened GitHub profile - not an "unspoofable" or "fully independent" guarantee. The recording is an edited screen recording assembled from real public GitHub browser captures (no synthetic UI). See the live evidence on the demo repo and pull requests #1, #2, #3. The recording plan, captions, shot list, and evidence manifest live in docs/media/.
Costs are calculated from a sourced multi-provider table - Anthropic (Opus / Sonnet / Haiku), OpenAI (GPT-5 family + legacy GPT-4), and DeepSeek (V4 Flash / V4 Pro) - with cache-read and batch discounts handled, labeled with source and verification date. When a model is unknown, Runcap says unknown_price rather than guessing.
DeepSeek matters because its API is OpenAI-compatible: point the gateway at https://api.deepseek.com with your DeepSeek key and Runcap prices, caps, and compresses it with zero extra setup - the same one command as OpenAI. At roughly $0.14 / $0.28 per million input/output tokens it is far cheaper than the US frontier models, so the people running the biggest agent loops on it are exactly the ones a hard cap protects.
Runcap is built not to fake certainty. Every important output carries a truth label:
observed- git diff, exit code, file changes, terminal output;calculated- parsed errors, diff hashes, stuck score, cost from the sourced price table;provider_usage- token usage returned by the upstream provider;manual_calibration- subscription % you entered before/after a run;unknown- Runcap cannot honestly know.
If it cannot prove something, it says so.
Runcap v0.6 is open-source and free under MIT.
The local CLI and CI adjudication mode are available now. Hosted sync, team budget pools, organization reporting, and paid plans are future ideas only. They are not available for purchase today.
A working local tool plus an optional CI adjudication mode, not a hosted SaaS. Ready for: wrapping real Codex / Claude / Cursor sessions, catching stuck agents, proving rescue prompts save time, and gating AI-generated pull requests in GitHub Actions. Not yet: a hosted cloud platform or a universal observability standard. It is not trying to replace Langfuse or LiteLLM; it focuses on a different layer - pre-run cost caps and merge-eligibility evidence.
Runcap is built and maintained by Kirill D., a solo AI and automation consultant based in Calgary, Canada. He helps solo SaaS founders and service businesses ship AI features that hold up in production - cost control, vibe-code audits, and reliable automation. More at launchsoloai.com.
The thesis: AI can propose a change. It should not certify its own success.