feat: drift alert self-calibrates to the install's own scatter - #22
Merged
Conversation
The degradation watch alarmed on a fixed 2.5 °C delta. On a real install with ~3.4 °C session-to-session scatter that sits near one sigma — about a 14% chance of firing per look with no real change — and the same value would be far too loose on a quiet install and far too tight on a noisy one. detect_drift already computed a Student-t confidence interval from the install's own MAD; it just wasn't the trigger. Now "drifting" (the alert) needs both: the interval clears zero, and the delta is material (>= DRIFT_WARN_C, demoted from trigger to floor). The effective threshold served as threshold_c is the larger of the floor and what the scatter demands, so the dashboard shows which one binds. A delta past the floor whose interval straddles zero is a "lead": dashboard chip, one default-priority push per episode (latched in the poller), an event — no alert row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The degradation watch fired on a fixed Δ ≥ 2.5 °C between the recent-3 median and the baseline median. That number was tuned to one install and is wrong everywhere else in both directions: on this install (~3.4 °C session-to-session scatter, MAD-σ 3.0) it sits at ~1.07σ, i.e. ~14% chance of an alert per look with no real change — the 08-12 phone alert was one of those. On a quieter install it is too loose; on a noisier one (variable ambient, a sensor in a draughty spot) it fires constantly.
detect_driftalready computed a Student-t 95% CI from the install's own MAD and reported it asconfident; it just wasn't the trigger.Follow-up to #21, which made leads push at default priority but still raised the alert row for them.
Code touched
wallmonitor/thermal.py—detect_drift:drifting(the alert) now requires bothconfident(CI clears zero) and Δ ≥DRIFT_WARN_C, which is demoted from trigger to materiality floor (a confirmed 0.3 °C increase is real but not worth an inspection). Newlead= past the floor but not confirmed.threshold_cnow reports the effective threshold —max(floor, t95 × SE(Δ))— andfloor_cthe constant, so the UI can say which one binds. Docstring rewritten.wallmonitor/poller.py—recheck_thermal_driftis a three-way branch: confirmed → alert row +thermal_driftevent + high-priority push (unchanged); lead → no alert row,thermal_drift_leadevent + one default-priority push per episode, latched in_drift_lead_active(previously the alert row's "newly raised" did the de-duplication); stable → clear. The lead push body now states what Δ this install needs to confirm.wallmonitor/static/app.js— dashboard: awarningchip "heat rise: lead" alongside the existingserious"heat rise increasing"; the "stable" model note is suppressed while a lead is open. Alerts page: Confirmed / Lead / Stable phrasing, and the threshold line says whether the floor or the scatter is binding.docs/thermal-model.md— new "What counts as drift" subsection.tests/test_wallmonitor.py—test_thermal_drift_threshold_follows_install_scatter: same Δ ≈ 3 °C on a quiet install → alert, on a noisy install → lead; a confirmed-but-tiny Δ → neither.wide_scattertest now asserts lead-not-alert; the poller lead test asserts no alert row, athermal_drift_leadevent, and that a second recheck does not push again.Risk
alertstable orthermal_driftevents (JSON webhook consumers, the dashboard banner) sees confirmed drift only. Leads live inthermal_drift_leadevents and the dashboard chip.settingsis possible if it annoys.DRIFT_WARN_C = 2.5remains a single-install judgement about what increase is worth an inspection. It is now a floor rather than the trigger, which is a much smaller role; a per-install materiality floor has no obvious data source.Verification
uv run pytest -q: 112 passed (111 before + 1 new; 3 existing tests updated to the new semantics).drifting: false, lead: true. Under this PR it is a dashboard chip and one default-priority push, no alert row. That is the self-calibration on real data: this install's scatter demanded 4.8 °C, the tripwire asked 2.5.Deploy: restart the service. No migration; the
leadlatch starts clear, so an open lead at restart pushes once.