live SSE relay for tools/call

[kam-validmind]

What and why?

The problem: when an agent calls a slow tool through Atryum, Atryum used to wait for the tool to completely finish, then send back one final answer. If the tool took 30 seconds, the agent saw and heard nothing for 30 seconds, then got the result all at once.

The fix: Atryum can now forward "still working, X% done" style updates to the agent while the tool is still running, instead of making it wait for one final answer. This only happens if the agent says it can handle that (Accept: text/event-stream) and the upstream tool server supports it — otherwise nothing changes, and the agent gets the same single response as before.

Before vs. after:

sequenceDiagram
    autonumber
    participant Agent
    participant Atryum
    participant Tool as Upstream tool server

    rect rgb(245, 235, 235)
    Note over Agent,Tool: Before this PR
    Agent->>Atryum: "run this tool"
    Atryum->>Tool: "run this tool"
    Note over Atryum,Tool: (silence for the whole run, could be minutes)
    Tool-->>Atryum: final result
    Atryum-->>Agent: final result
    end

Loading

sequenceDiagram
    autonumber
    participant Agent
    participant Atryum
    participant Tool as Upstream tool server

    rect rgb(230, 245, 235)
    Note over Agent,Tool: After this PR (only if Agent asks for it)
    Agent->>Atryum: "run this tool, I can read a live stream"
    Atryum->>Tool: "run this tool"
    loop while the tool keeps working
        Tool-->>Atryum: "33% done…"
        Atryum-->>Agent: "33% done…"
    end
    Tool-->>Atryum: final result
    Atryum-->>Agent: final result
    end

Loading

A quick glossary, since this touches a protocol most of the codebase doesn't deal with directly:

Term	Plain-English meaning
MCP	Model Context Protocol — the standard agents use to call tools. Messages are JSON.
Upstream	The tool server doing the actual work (e.g. a GitHub MCP server).
Downstream	The agent/agent harness that's waiting on Atryum for an answer.
SSE (Server-Sent Events)	A way for a server to keep one HTTP connection open and send multiple small messages over time, instead of one message and done. This is what makes "live updates" possible over plain HTTP.
Progress notification	A "still working, here's how far along I am" message. It's one-way — no reply expected.
Terminal response	The actual final answer to the tool call. There's exactly one of these per call, streaming or not.
Progress token	An ID the agent hands Atryum so progress updates can be matched back to the right call.

Where the progress updates can come from. Most upstream servers put progress messages on the same HTTP response as the tool call itself ("Path A" below). A few SDKs (notably the reference Python MCP SDK) instead send progress on a second, separate connection shared by every in-flight call ("Path B"). Atryum listens on both, so either kind of upstream server works:

flowchart TB
    Downstream[Agent]
    Relay[Atryum]
    Upstream[Upstream tool server]
    CallPath["Path A: the response to this one tool call<br/>carries progress AND the final answer"]
    StandalonePath["Path B: one shared connection<br/>carries progress ONLY, for all active calls"]

    Downstream <-->|"tool call"| Relay
    Relay --- CallPath
    CallPath --- Upstream
    Relay --- StandalonePath
    StandalonePath --- Upstream

Loading

Because Path B is shared across every call currently in flight, Atryum has to make sure one caller's progress updates never leak to a different caller. It does this by swapping in a unique progress token for each call before forwarding it upstream, then swapping the caller's original token back in before relaying the update — so two agents that happen to pick the same token number can never see each other's progress.

Safety nets included in this PR (see docs/architecture.md → "Live SSE relay for tools/call" for the full write-up, and CHANGELOG.md for the complete list):

• A kill switch — stream_relay_enabled = false turns this whole feature off and restores the old one-response-at-the-end behavior, no rollback needed.
• Timeouts for "upstream went quiet too long," "upstream took too long overall," and "one message was too big" — so a broken or hostile upstream can't hang a request forever or blow up Atryum's memory.
• If the connection to a well-behaved upstream drops mid-stream, Atryum reconnects and picks up where it left off instead of restarting or losing progress.
• Every progress message and the final result get written to Atryum's audit log (invocation_events), same as before — reviewers/admins don't lose visibility just because delivery got more real-time.
• Side fix, unrelated to streaming but found while working on this: if an agent disconnected mid-call on a normal (non-streaming) tool call, the invocation could get stuck showing executing forever in the audit log, even though it had actually finished. That's fixed too.

How to test

• go test ./... (or just check) — runs the full new test suite with no network access needed. If you want to look at specific pieces: internal/mcp/http_stream_test.go (Path A), internal/mcp/standalone_stream_test.go (Path B), internal/invocation/stream_execution_test.go and stream_sink_internal_test.go (audit logging), internal/api/sse_relay_test.go (the HTTP-facing relay).
• Two extra tests run against real MCP servers instead of fakes. They're opt-in (skipped by default, and skip themselves cleanly if the tool below isn't installed) so CI stays fast:
  • just mcp-everything-test — spins up the official @modelcontextprotocol/server-everything reference server and calls its trigger-long-running-operation tool through a real Atryum server. Needs npx.
  • just mcp-standalone-stream-test — spins up a small Python MCP server (internal/api/testdata/mcp_standalone_fixture/server.py) to test the shared-connection (Path B) case specifically. Needs uv.
• To poke at it by hand: configure an [[mcp_servers]] entry pointing at either server above, call one of its slow tools with Accept: text/event-stream set on the request, and watch invocation_events fill in with multiple stream_event rows as it runs instead of a single row at the end.

What needs special review?

• internal/api/sse_relay.go + internal/invocation/stream_sink.go — this is where "don't lose progress, but also don't let a slow audit write block a live update" is implemented. Trickiest part of the PR.
• internal/mcp/standalone_stream.go — the progress-token swap described above, plus what happens when the upstream session gets renewed mid-flight.
• The three-way distinction between stream_canceled / stream_aborted_downstream / stream_timeout in the audit log — worth checking the reasoning holds up. Background in the fix: address stream relay review findings and finalize buffered calls after disconnect; harden relay lifecycle commit messages.
• Whether the new default timeout/size values in atryum.example.toml are reasonable for real-world upstreams.

Dependencies, breaking changes, and deployment notes

• No breaking changes for existing tool calls — streaming only turns on when both the agent asks for it and the upstream supports it. Everything else behaves exactly as before.
• stream_relay_enabled can disable the new behavior at startup, if something unexpected shows up with a production upstream.
• No new dependencies ship in the binary. The two real-server tests need npx/uv on the machine running them, but they're excluded from the default go test ./... / CI run and skip themselves gracefully if those tools are missing.

Data impact

• Yes — this change is data-impacting
• No — this change is not data-impacting

Justification: No existing customer data, config format, or API response shape changes. The only new persisted data is additional audit rows (stream_event, stream_completed, stream_delivery in invocation_events) for calls that use streaming — purely additive telemetry, nothing existing is modified or removed.

Release notes

Atryum can now stream live progress updates from a tool call back to the calling agent as they happen, instead of making the agent wait silently for one final answer. This works whether the upstream tool server sends progress on the tool call's own response or on a separate shared connection (used by some SDKs, including the reference Python MCP SDK). See the [Unreleased] section of CHANGELOG.md for the new configuration options and full behavior guarantees.

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• Data impact classified
• PR linked to Shortcut
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required) - put it in a markdown file