Skip to content

feat: drift alert self-calibrates to the install's own scatter - #22

Merged
zebraengine merged 1 commit into
mainfrom
feat/drift-self-calibrating
Aug 27, 2026
Merged

feat: drift alert self-calibrates to the install's own scatter#22
zebraengine merged 1 commit into
mainfrom
feat/drift-self-calibrating

Conversation

@zebraengine

Copy link
Copy Markdown
Owner

Problem

The degradation watch fired on a fixed Δ ≥ 2.5 °C between the recent-3 median and the baseline median. That number was tuned to one install and is wrong everywhere else in both directions: on this install (~3.4 °C session-to-session scatter, MAD-σ 3.0) it sits at ~1.07σ, i.e. ~14% chance of an alert per look with no real change — the 08-12 phone alert was one of those. On a quieter install it is too loose; on a noisier one (variable ambient, a sensor in a draughty spot) it fires constantly. detect_drift already computed a Student-t 95% CI from the install's own MAD and reported it as confident; it just wasn't the trigger.

Follow-up to #21, which made leads push at default priority but still raised the alert row for them.

Code touched

  • wallmonitor/thermal.pydetect_drift: drifting (the alert) now requires both confident (CI clears zero) and Δ ≥ DRIFT_WARN_C, which is demoted from trigger to materiality floor (a confirmed 0.3 °C increase is real but not worth an inspection). New lead = past the floor but not confirmed. threshold_c now reports the effective threshold — max(floor, t95 × SE(Δ)) — and floor_c the constant, so the UI can say which one binds. Docstring rewritten.
  • wallmonitor/poller.pyrecheck_thermal_drift is a three-way branch: confirmed → alert row + thermal_drift event + high-priority push (unchanged); lead → no alert row, thermal_drift_lead event + one default-priority push per episode, latched in _drift_lead_active (previously the alert row's "newly raised" did the de-duplication); stable → clear. The lead push body now states what Δ this install needs to confirm.
  • wallmonitor/static/app.js — dashboard: a warning chip "heat rise: lead" alongside the existing serious "heat rise increasing"; the "stable" model note is suppressed while a lead is open. Alerts page: Confirmed / Lead / Stable phrasing, and the threshold line says whether the floor or the scatter is binding.
  • docs/thermal-model.md — new "What counts as drift" subsection.
  • tests/test_wallmonitor.pytest_thermal_drift_threshold_follows_install_scatter: same Δ ≈ 3 °C on a quiet install → alert, on a noisy install → lead; a confirmed-but-tiny Δ → neither. wide_scatter test now asserts lead-not-alert; the poller lead test asserts no alert row, a thermal_drift_lead event, and that a second recheck does not push again.

Risk

  • Fewer alert rows, by design. Anything watching the alerts table or thermal_drift events (JSON webhook consumers, the dashboard banner) sees confirmed drift only. Leads live in thermal_drift_lead events and the dashboard chip.
  • The lead latch is in-process. A service restart while a lead is open re-pushes it once at default priority. Acceptable for a quiet notification; persisting the latch in settings is possible if it annoys.
  • Small samples make the CI wide by construction (t-multiplier 4.3 at df=2), so a genuinely degrading install with only six fits will sit as a lead for a few more sessions before it alarms. That is the honest position — the old tripwire would have alarmed on the same data, but also on scatter. The lead push says what Δ it needs, so the user is not left guessing.
  • DRIFT_WARN_C = 2.5 remains a single-install judgement about what increase is worth an inspection. It is now a floor rather than the trigger, which is a much smaller role; a per-install materiality floor has no obvious data source.

Verification

  • uv run pytest -q: 112 passed (111 before + 1 new; 3 existing tests updated to the new semantics).
  • Replayed read-only against the production DB on the mini PC:
    • 08-12 verdict (the alert that reached the phone): Δ 3.72, CI [−1.1, +8.6], effective threshold 4.83 °Cdrifting: false, lead: true. Under this PR it is a dashboard chip and one default-priority push, no alert row. That is the self-calibration on real data: this install's scatter demanded 4.8 °C, the tripwire asked 2.5.
    • Live verdict now: Δ 0.74, CI [−3.3, +4.8], threshold 4.06 → nothing. Note the threshold already tightened from 4.83 to 4.06 as the baseline grew 5 → 11 fits, which is the behaviour a user should expect: the watch gets sharper with history.

Deploy: restart the service. No migration; the lead latch starts clear, so an open lead at restart pushes once.

The degradation watch alarmed on a fixed 2.5 °C delta. On a real install
with ~3.4 °C session-to-session scatter that sits near one sigma — about
a 14% chance of firing per look with no real change — and the same value
would be far too loose on a quiet install and far too tight on a noisy
one. detect_drift already computed a Student-t confidence interval from
the install's own MAD; it just wasn't the trigger.

Now "drifting" (the alert) needs both: the interval clears zero, and the
delta is material (>= DRIFT_WARN_C, demoted from trigger to floor). The
effective threshold served as threshold_c is the larger of the floor and
what the scatter demands, so the dashboard shows which one binds. A delta
past the floor whose interval straddles zero is a "lead": dashboard chip,
one default-priority push per episode (latched in the poller), an event —
no alert row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@zebraengine
zebraengine merged commit bb20ca4 into main Aug 27, 2026
1 check passed
@zebraengine zebraengine mentioned this pull request Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant