fix(scoring): bound score queue and batch payloads

[kxzk]

TL;DR

Asynchronous score batching now bounds memory use and limits multi-score request size.

Why

An ingestion outage could grow the score queue without a limit. Large flushes could also exceed the ingestion request limit.

Checklist

• Has label
• Has linked issue
• Tests added for new behavior
• Docs updated (if user-facing)

Closes #

Note

Kade's words:

Summary

Asynchronous score batching now keeps at most 100,000 pending scores by default. Applications can set score_queue_capacity. A full queue logs an error and drops the new asynchronous score without waiting for capacity. Synchronous score creation stays unchanged.

Flushes split before a multi-score JSON payload exceeds 2.5 MB. Delivered batches leave the queue. A failed batch and all later scores stay queued in their original order. This design keeps retry memory bounded.

Change diagram

flowchart TD
    accTitle: Bounded asynchronous score delivery
    accDescr: New scores enter a bounded queue. Flushes send size-limited batches and retain failed work in order.
    C["Create asynchronous score"] --> Q{"Queue has capacity"}
    Q -->|"No"| D["Log error and drop new score"]
    Q -->|"Yes"| A["Append score"]
    A --> B["Build payload below 2.5 MB"]
    B --> S{"Batch delivery succeeds"}
    S -->|"Yes"| R["Remove delivered prefix"]
    S -->|"No"| K["Keep failed batch in order"]

Loading

Verification

I ran the complete RSpec suite locally. All 1,578 examples passed. Line coverage was 97.33%.

I ran RuboCop against 98 files. RuboCop reported no offenses.

I ran a live Ruby SDK probe with a forced small payload limit. The SDK sent six accepted scores in six separate ingestion requests for trace b30020337050ed408b8f27469c4b4d2c. The Langfuse CLI read back the trace and all six score names.

I did not send a production-sized 2.5 MB payload to Langfuse. Unit tests validate exact JSON byte accounting at the split boundary. I did not preview the rendered Mermaid diagram in GitHub. I reviewed the Mermaid source and syntax locally.

---

Note

Medium Risk
Changes async score delivery, drop-on-full, and retry/discard semantics—scores can be lost under overload or permanent API rejection, though behavior is documented and heavily tested.

Overview
Asynchronous score batching is now memory-bounded and safer under API failures. Apps can set score_queue_capacity (default 100,000); when the queue is full the SDK logs and drops only the new async score—no blocking. Sync create_score! is unchanged.

Flush behavior replaces draining the whole unbounded queue with ordered prefix delivery: multi-score requests are split before ~2.5 MB JSON; successful prefixes are removed; retryable failures (429/5xx, network, auth) leave the failed batch at the front; non-retryable batch errors drop that batch and continue. Score bodies are JSON-snapshotted at enqueue so caller mutation cannot corrupt pending events.

Ingestion batch failures now raise BatchDeliveryError with a retryable? flag instead of generic ApiError where appropriate.

^{Reviewed by Cursor Bugbot for commit a767bbb. Bugbot is set up for automated code reviews on this repo. Configure here.}

[linear-code]

AAI-390

[kxzk]

cursor review verbose=true

[cursor]

Bugbot request id: serverGenReqId_8f9c9ada-14fb-4aec-8105-c2224c85c660

[cursor]

Bugbot rules debug

No rules were used for this review.

https://cursor.com/docs/bugbot#team-rules

Bugbot request id: serverGenReqId_8f9c9ada-14fb-4aec-8105-c2224c85c660

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 66c7d9c. Configure here.}