Skip to content

Add incident-report skill - #32

Open
JornC wants to merge 4 commits into
aerius:mainfrom
JornC:incident-report-skill
Open

JornC wants to merge 4 commits into
aerius:mainfrom
JornC:incident-report-skill

Conversation

@JornC

@JornC JornC commented Sep 21, 2026

Copy link
Copy Markdown
Member

This adds a skill and guide for how to figure out how and why a particular flake is flaking, in order to get at the root cause, and fix it at the core instead of papering over the top of it.


LLM-generated technical summary

Adds agentic-skills/incident-report.md and a row for it in the agentic-skills README table.

The skill covers two things: investigating a failed or flaky run (test, deploy or environment) down to its mechanism, and writing that up as a blame-free report. It is written as a guide to pick from rather than a fixed procedure, and says so up front; the developer driving the investigation can narrow or reorder it.

The investigation part lists ways in: pin the exact failure signatures and when each wait started, check which commit the run actually tested, put CI output, deploy output, container logs, metrics, config and code on one timeline, diff against the last green run, test the explanations already on the table against the records, find the mechanism in config and code, reproduce locally, and measure how often with margins rather than pass/fail. It asks for the trigger to be explained and not only the gap, and for measured event counts to win over conclusions drawn from how the config ought to behave.

The report part separates certain claims (read directly from a record) from inferred ones (a conclusion, with what it rests on and a confidence in words, never an unmeasured percentage), allows one labelled best guess when the cause stays unknown, and gives an optional section list: TL;DR, timeline, mechanism, comparison with green runs, reproduction, frequency, considered and ruled out, fix, what does not fix it, open questions. The fix section is limited to what would have prevented the incident, and the checklist before publishing covers names, secrets copied from logs, and the impact outside test environments.

The content is drawn from a handful of recent incident write-ups and the investigations behind them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant