Capture MCP tool calls in PostHog - #128
Open
masnwilliams wants to merge 8 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
masnwilliams
marked this pull request as ready for review
July 29, 2026 19:11
masnwilliams
marked this pull request as draft
July 29, 2026 19:16
masnwilliams
marked this pull request as ready for review
July 29, 2026 19:27
masnwilliams
marked this pull request as draft
July 29, 2026 19:35
masnwilliams
marked this pull request as ready for review
July 29, 2026 19:36
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d9b21ea. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Instruments the MCP server with PostHog MCP analytics (
@posthog/mcp, pinned to0.10.1) so every inboundtools/call,tools/list, andinitializeis captured as a$mcp_*event — tool name, latency, error state, client name/version, session.What that buys us: per-tool call volume, error rate and latency percentiles, which clients connect, and which advertised tools never get called. Today we ship tool changes blind — kernel API traffic tagged
X-Source: mcp-servershows endpoints, not tools.src/lib/mcp/analytics.ts— module-scopeposthog-nodeclient plusinstrument()wiring.identifyattributes events to the Clerk user id when the request carries a JWT; API-key requests stay session-scoped rather than getting a made-up identity.src/app/[transport]/route.ts— instrument inside thecreateMcpHandlercallback, mint a session id on the handshake (below), and drain the queue withafter()so the flush runs once the response is already on the wire. Capture adds no latency to a tool call..env.example—POSTHOG_PROJECT_TOKENandPOSTHOG_HOST. With no token set the whole thing is a no-op, so deploys and local dev without PostHog credentials are unaffected (dev logs a one-time error so the miss isn't silent).POSTHOG_PROJECT_TOKENandPOSTHOG_HOSTare set on the Vercel project (production → prod PostHog project, preview → staging).Sessions
mcp-handlerruns the streamable-HTTP transport statelessly and answers over SSE, so it never issues anMcp-Session-Id. Left alone that means one PostHog session per HTTP request, and$mcp_client_name/$mcp_client_version— only sent atinitialize— missing from every later event.mintMcpSessionIdhandles it the way the SDK documents for SSE: on aninitializePOST it mints a session token carrying a fresh session id plus the client name and version, injects it on the inbound request (so the handshake event lands in that session too) and echoes it back on the response. Clients replay the header, and any instance decodes the same values out of it — no session store, no sticky routing. The stateless transport ignores an incoming session id (validateSessionshort-circuits whensessionIdGeneratoris undefined), so this can't produce 400/404s.Mcp-Session-Idis added to the CORS allow/expose lists so browser-based clients can round-trip it.No payloads leave the server
beforeSenddrops$mcp_parametersand$mcp_responseon every event.Neither side of a call is safe to capture here. Results are serialized through
jsonResponse, so a response reaches the SDK as one JSON string and its key-name redaction can't see inside it — a newly created API key, a TOTP code, or a CDP URL would go out verbatim. Arguments are free-form on many tools: credential field maps (manage_credentialsvalues,manage_auth_connectionsfields),browser_curlheaders and body,computer_actiontyped text,exec_commandcommands,execute_playwright_codesource. An allow-list of "credential tools" would always be one tool behind, so the payloads go entirely and what remains is call metadata.If we later want an argument breakdown (e.g. which
actionis most used), the safe shape is an explicit allow-list of individual parameter keys.Deliberately left off
instrument()can inject a requiredcontextargument into every tool schema to capture what the agent was trying to do ($mcp_intent). It's the most interesting signal, but it changes a public tool surface — a required field plus a 15-25 word instruction on all 17 toolsets — so this ships withcontext: false. Worth turning on as its own change once the basic numbers are landing.The SDK is pre-1.0 (
0.x), so event names and properties can change in minor releases — hence the exact-version pin.Verification
bunx tsc --noEmitpasses (what CI runs).tools/listand atools/callreplaying the minted header land as three events under one session id — the exact id inside the token — each carrying client name and version. A control request that doesn't replay the header lands in its own session with a null client, which is what the currently deployed staging server produces for every event.$mcp_parametersnor$mcp_responsepresent, and no canary string anywhere in the event.bun run buildfails locally on this branch and onmainalike (page data collection for/authorizewithout Clerk/Redis env), so that failure is unrelated.Note
$mcp_server_namecurrently readsmcp-typescript server on vercel— mcp-handler's defaultserverInfo, which is also what agents see. PassingserverInfo: { name: "kernel-mcp-server", version }tocreateMcpHandlerwould fix both, but it changes the advertised server identity so it's left out of this PR.Note
Medium Risk
Touches the authenticated MCP request path and CORS for a new header; payload stripping reduces data-leak risk but misconfiguration could still send events externally when a token is set.
Overview
Adds optional PostHog MCP analytics so
initialize,tools/list, andtools/callemit$mcp_*events (tool, latency, errors, client, session) without changing tool schemas. Capture is a no-op whenPOSTHOG_PROJECT_TOKENis unset;.env.exampledocumentsPOSTHOG_PROJECT_TOKENandPOSTHOG_HOST.New
src/lib/mcp/analytics.tswires@posthog/mcpinstrument()with Clerk distinctId for JWT requests,beforeSendstripping of parameters and responses, andcontext: falseso tools stay unchanged. The MCP route callsinstrumentMcpAnalyticsin the handler factory and uses Nextafter(flushMcpAnalytics)on GET/POST so flushes run after the response.Because the stateless transport does not issue
Mcp-Session-Id,mintMcpSessionIdoninitializePOST injects and echoes the SDK session header (with CORS allow/expose) so clients can correlate handshake and later calls in one PostHog session.Reviewed by Cursor Bugbot for commit 1de2e9a. Bugbot is set up for automated code reviews on this repo. Configure here.