vibe-check report + specific finding: xAI prompt-cache hits dropped in getBatchUsage (agent.ts:2321)

[mlops-kelvin]

Hi — I'm Kelvin, MLOps on the Marbell team. One of the things we maintain is vibe-check, a small Docker-based static analysis harness that runs ten tools (ruff, pyright, lizard, radon, pyscn, deepcsim, wily, eslint, tsc, jscpd) and aggregates them into an A–F grade across six dimensions. We also maintain claude-code-xai, a Claude-Code-to-xAI bridge, so we spend a lot of time in the same wire protocols grok-cli talks to.

We ran vibe-check against commit 8b148ac of grok-cli because we were curious how another team approaches the same xAI Responses API surface we work against. Sharing the report plus one specific finding that we think is worth a quick PR.

Headline: Overall grade F (39/100). The real story is more nuanced than the grade suggests — duplication and linting are both excellent, and the worst complexity outliers are concentrated in the UI layer, not the agent core. But there's one specific gap in the usage-parsing path that is probably costing grok-cli users real money.

Full report

Dimension	Score	Grade
Linting	0 issues	A
Type Safety	2050 errors, 0 warnings	F
Complexity	avg CC 3.3, max CC 553	D
Maintainability	no data	D
Duplication	2.2% duplicated (40 clones)	A
Hygiene	75/100 (license / tests / README / .gitignore all present; secrets scan flagged potential matches)	B

Auto-F triggers: potential hardcoded secrets, and one anonymous function in app.tsx at CC=553.

The most actionable finding: xAI prompt-cache hits are silently dropped in getBatchUsage

src/agent/agent.ts:2321 defines the usage accumulator:

function getBatchUsage(response: BatchChatCompletionResponse): ProcessMessageUsage {
  const usage = response.usage ?? {};
  const inputTokens = asNumber(usage.input_tokens) ?? asNumber(usage.prompt_tokens);
  const outputTokens = asNumber(usage.output_tokens) ?? asNumber(usage.completion_tokens);
  const totalTokens = asNumber(usage.total_tokens) ?? sumDefined(inputTokens, outputTokens);
  return {
    inputTokens,
    outputTokens,
    totalTokens,
    costUsdTicks: asNumber(usage.cost_in_usd_ticks),
  };
}

This parses total tokens and cost, but never reads usage.input_tokens_details.cached_tokens (the Responses API path) or usage.prompt_tokens_details.cached_tokens (the Chat Completions path). xAI's prompt cache gives a 90% discount on cached input tokens, and the Responses API surfaces cache hits in exactly those fields. Because getBatchUsage stops at the top-level input_tokens, grok-cli:

1. Can't show cache savings to users — the totalTokens counter is the true total cached+uncached, but there's no breakdown.
2. Undercounts savings in session logs — a session with heavy system-prompt reuse (the normal case for a coding agent) leaves most of its discount invisible.
3. Doesn't differentiate cache-hit sessions from cache-miss sessions for debugging — e.g. when a prompt change accidentally busts the cache and costs jump 10×.

The fix is small and isolated to this function. For reference, the equivalent read in our bridge lives at translation/responses_reverse.py:40-50 and looks like:

usage = response.get("usage", {})
prompt_details = usage.get("input_tokens_details", {})
cached_tokens = prompt_details.get("cached_tokens", 0)

We'd be glad to send a PR adding this to getBatchUsage, plumbing the field through ProcessMessageUsage and accumulateUsage, and surfacing it in the session UI / telemetry wherever totalTokens is currently shown. Small, scoped, clearly bounded.

Other real findings

1. Type safety — 2050 pyright errors. biome is configured and the lint count is zero, which is admirable, but pyright (or tsc --noEmit, which is already in package.json as typecheck) is reporting a lot. A ratchet in CI — freeze the current count, don't let it rise — would prevent this from getting worse while you chip away at it.

2. src/ui/app.tsx complexity is concentrated and load-bearing. The top complexity hotspots are:

   • Anonymous function in app.tsx at CC=553 (this is what auto-triggered the F)
   • SessionHeader in app.tsx at CC=139
   • Anonymous function in app.tsx at CC=114
   • Four more anonymous functions in app.tsx at CC=57/43/43/38

   These are React components; CC=553 on an anonymous component suggests a very large inline render function. That's painful to test, painful to reason about, and painful to refactor later. Worth extracting subcomponents — each extracted piece drops its CC to something testable in isolation. This is also the kind of area where "built with Cursor" tends to produce organic sprawl: the first draft works, so it never gets broken up.

3. src/agent/agent.ts::toBatchChatMessages CC=141. This is the function that converts internal messages to the batch-API format — on the hot path of every agent turn. High CC here is the same flavor as KIRA's _run_agent_loop (which we looked at separately): it's doing real work handling edge cases, but at 141 it's crossed the threshold where testing individual branches becomes very hard. Worth splitting by message type (user / assistant / tool-result / tool-call) into named helpers.

4. src/utils/environment.ts::readJson CC=77. A JSON reader at CC=77 almost always means the function is doing validation + migration + defaulting inline. Extracting the validation and migration steps would make this testable and make errors from specific phases attributable.

5. Potential secret flagged by the hygiene scan. Regardless of our report, this is the one finding to triage first. Happy to share the matcher if helpful.

6. Duplication is excellent at 2.2%. Only real duplication is in the test files (orchestrator.test.ts ↔ runtime-prep.test.ts), which is expected test scaffolding, and two small blocks between entrypoint.ts / environment.ts / recipes.ts. Worth knowing this is not the problem area — the codebase is genuinely not copy-pasted.

One note on the SDK choice

We expected to find raw HTTP calls to /v1/responses based on prior context, and instead found grok-cli using @ai-sdk/xai (Vercel AI SDK) via createXai in src/grok/client.ts, with provider.responses(modelId) for models flagged responsesOnly: true. That's a sensible TypeScript path — the official xai-sdk is Python-only, and Vercel AI SDK's xAI provider is the best-maintained TypeScript abstraction over the Responses API today. For contrast, our own bridge (written before the Vercel AI SDK's xAI provider stabilized) uses httpx.AsyncClient directly against https://api.x.ai/v1. Both paths work; yours is arguably more future-proof for new xAI features.

The tradeoff worth knowing about: Vercel AI SDK normalizes the usage object to inputTokens/outputTokens/totalTokens, so the raw input_tokens_details.cached_tokens field is not always surfaced at the top level of generateText/streamText results — you have to reach for providerMetadata or the raw response. That may be part of why this got missed. Happy to scope the cache-token PR around whatever the cleanest access path is in your current SDK version.

Happy to help with

Scoped PRs, only the ones you want:

• (a) Adding cachedInputTokens to getBatchUsage / ProcessMessageUsage / accumulateUsage and surfacing it in the UI where total tokens are shown. Highest leverage — unblocks cost observability and is the most direct user-visible win.
• (b) A pyright ratchet in CI that freezes the current 2050-error count so it only drops from here.
• (c) Breaking up the top 3 app.tsx hotspots into subcomponents with unit tests.
• (d) A toBatchChatMessages split by message type.
• (e) Re-running vibe-check with verbose output and filing a targeted follow-up on whatever the secrets scan matched.

Caveat

We ran this as outsiders — you have context we don't. vibe-check's default recommendation at an F ("walk away or plan a complete rewrite") is the tool's generic boilerplate and explicitly not our read of grok-cli. 2856 stars, active maintenance, and a real product — there's a lot working here. We're sharing the report because the cache finding in particular is a specific user-visible cost issue that's easy to fix, and because static analysis is usually more useful than no signal at all.

If any of this misfires or isn't useful, close the issue and no hard feelings.

[homanp]

@mlops-kelvin thanks for the report, feel free to PRs to mitigate the issues you feel are important to fix. Thank you!