Not model quality — the plumbing around it. Retries that charge twice, provider calls with no deadline, agent loops with no ceiling, spend nobody capped, and prompt changes that quietly degrade output until a customer notices.
Same failure classes as any distributed system, wearing an AI hat.
10 ways your AI feature breaks in production → The full checklist I audit against. Free, no signup.
changedetection.io (33.6k★) — reviewed the LLM client's retry handling in #4383. The maintainer called the points "spot on" and shipped Retry-After handling per RFC 9110, backoff jitter and a delay ceiling in #4384; the third point was traced and closed as already safe.
Follow-ups: regression tests for the retry-budget behaviour that fix introduced (open), and #4385 — the AI intent filter fails closed on a malformed model response, suppressing the change and caching that verdict, while every other failure path in the same function fails open.
stripe-fulfilment-kit — fenced, leased order claims that survive retries, crashes and concurrent webhook deliveries. Cross-process atomic via SQLite. 59 tests, all offline.
ai-review-gate — two models review a coding agent's output and block it before it ships. Includes an honest record of the eight real errors it caught in one session.
cardamom-quant — quantitative model, 37-trial ledger.
A lease that passed 78 tests and was still wrong. A provider call that outlived its two-minute lease wrote its failure over a completed order — hiding a product the customer had paid for, and re-opening the order for a third charge.
Every test settled an order from its current owner. None covered a write from a superseded one, which is precisely the state a lease exists to create.
Full write-up — how 78 tests missed it →
Available for contract work · gg5533.github.io · samihabbal5@icloud.com