Nothing will watch Saruman's guests, and nothing here can tell a quiet lab from a dead one

[Gerrrt]

Since 2026-09-02, Saruman's own host agent reports into the estate's stack —
#88,
#240. The hypervisor is now
visible on VLAN 99. Its guests are not, and will not be: ADR-0007 keeps guest
telemetry in the lab, the NOTE on that ADR says so explicitly, and that part
is not being argued with here.

What follows from it has not been decided, and it is a silent failure in both
directions.

What the machines say

Read from oracle against the live stores, 2026-09-04.

• Saruman pushes two jobs, Saruman-metrics and Saruman-alloy, and journal
  logs. It ships no container or guest metrics — docker.alloy only loads
  where there is a Docker socket, and the native runtime was chosen here.

• The estate's only net for a host that goes quiet is RemoteWriteJobStale,
  and it keys on jobs that arrive here:

  count by (job, instance) (count_over_time(up{job=~"integrations/.+|.+-metrics|.+-alloy"}[1d]))
    unless count by (job, instance) (up{job=~"integrations/.+|.+-metrics|.+-alloy"})
  

  That regex is why Saruman was covered the moment it started pushing,
  without a rule change. It is also why nothing that stays in the lab can ever
  be covered by it.

• Nothing is firing or pending for Saruman today. It is healthy, and that is
  the point below.

The gap

Before stacks/lab/ exists, a guest is monitored by nothing at all. Not by
the estate, by design; not by the lab, because #101
records that stacks/ contains exactly one directory and none of the lab is
built. ADR-0007 wants the Windows domain, Wazuh and Velociraptor on this host,
and any of them could be stood up before the second stack is. The window is
open now.

After it exists, nothing here can tell a quiet lab from a dead one. The
hypervisor's agent keeps reporting a healthy DL360 either way — that signal is
what makes the failure look fine. And the lab stack as specified is Loki,
Grafana and Alloy: no Alertmanager. ADR-0007 names giving up "Loki, the
dashboards and 40 alert rules" as the cost of the isolation, and the 40 rules
are the half nothing replaces. A log store and a dashboard are things a person
looks at; neither pages anyone.

This estate already writes rules about exactly this failure mode.
RemoteWriteJobStale's own description says it: "a dead agent stops pushing
rather than reporting a failure, so it goes quiet instead of going red." The
lab is being built to go quiet.

Worth deciding

Not a proposal — the isolation is sound and ADR-0007 should not be reopened
for it. The question is whether liveness is the same thing as telemetry.

• Let a heartbeat cross, not data. The pass 10.0.30.110 → 10.0.99.20:9090,3100 is already open and already unlogged. A handful of
  series from the lab stack — is it up, when did it last ingest — would be
  caught by the existing regex if the job is named *-metrics or *-alloy,
  with no new rule and no new firewall pass. It ships nothing about what the
  guests are doing, which is what ADR-0007 is protecting.
• Or give the lab its own Alertmanager and a route out of the segment.
  That is a second set of alert routes and age keys to look after, which is the
  cost ADR-0007: build stacks/lab/ on Saruman — nothing of it exists #101 already flags for the second stack, plus a rule that does not exist
  yet.
• Or accept it and write it down. A lab whose monitoring can die unnoticed
  may be a fine trade for a machine that is powered off between sessions —
  but then it should be a recorded decision rather than a consequence nobody
  chose.

Whichever way it goes, the near-term half is separable and cheap: decide what
watches a guest created before the lab stack exists.

Not settled here

Whether any of this changes ADR-0007. It probably does not — the ADR's Decision
is about telemetry, and a heartbeat about the stack itself may not be telemetry
in that sense. If it is, the amendment pattern from #88 already exists to
follow.

---

Filed from Lemmiwinks #108 /
PR #129, which corrected the
wiki pages that still said this host ran no agent. Sibling of
#101 (build stacks/lab/),
which owns the stack itself; this owns what watches it.