Skip to content

✨ feat(agents): token badge shows ↑context ↓output (bounded, not a 22M cumulative total) - #44

Merged
winlp4ever merged 2 commits into
mainfrom
feat/agent-tokens-metric
Jul 31, 2026
Merged

✨ feat(agents): token badge shows ↑context ↓output (bounded, not a 22M cumulative total)#44
winlp4ever merged 2 commits into
mainfrom
feat/agent-tokens-metric

Conversation

@winlp4ever

Copy link
Copy Markdown
Contributor

Follow-up to #43 (merged). Live testing showed a long session's badge reading 22M tok — that was the cumulative sum of input across every turn: technically real, but alarming and not what Claude Code shows. This reshapes the badge to the two numbers you'd actually want, matching Claude's ↑↓.

Change

TokenUsage becomes two meaningful values:

  • ↑ context = the latest turn's input (input + cache_read + cache_create) = how full the context window is right now. Bounded (~≤ the model window), not cumulative.
  • ↓ output = cumulative output generated (monotonic "how much this agent produced").

addLine now overwrites context to each assistant line's input and accumulates output, so after a read context reflects the last turn. Badge renders e.g. ↑148k ↓1.2k (tooltip: ↑ 148k context · ↓ 7.4M generated). Dropped the now-dead cumulative-sum path + addUsage.

Perf (measured, since it was asked)

The transcript read stays off the terminal hot path — main-process hook channel, fs.promises on the libuv threadpool, incremental by byte-offset, and parsed in bounded slices that await setImmediate between them. Benchmarked on a real 90.9 MB / 29,647-line transcript:

Worst single event-loop stall
Un-chunked (naive) 318 ms
1 MiB slices 13 ms
256 KiB slices (this PR) ~4.6 ms

~4.6 ms is under a third of a 16 ms frame, and only on the first read of a resumed session (a Stop, seconds-apart). Steady-state incremental reads (a turn's new KB) are sub-millisecond. Reduced the slice size 1 MiB → 256 KiB for the comfortable margin.

Tests

make check + typecheck green — 427 tests. Updated: addLine/TranscriptTokens assert the new {context, output} (incl. context-tracks-latest, output-accumulates, chunk-boundary + multibyte offset correctness); tokenBreakdown/badge assert ↑…k ↓…k.

The old badge summed input across every turn, so a long session read as "22M tok" —
technically the total processed, but alarming and not what Claude Code shows. Reshape
TokenUsage to two meaningful numbers:
  ↑ context = the LATEST turn's input (input + cache read + cache create) = how full the
    context window is right now (bounded, ~≤ the model's window)
  ↓ output  = cumulative output generated (a monotonic "how much this agent produced")
addLine now overwrites context to each assistant line's input and accumulates output, so
after a read context reflects the last turn. Drops the now-unused cumulative sum + addUsage.
@winlp4ever
winlp4ever merged commit 43999c6 into main Jul 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant