Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .workhorse/specs/tamanu/subjects.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ Which duties it can have, which facts describe it, and which checks apply to it

An agent reports an mSupply application, of type `msupply`, on a machine where mSupply is installed.

An agent reports a Tupaia application, of type `tupaia`, on a machine where Tupaia is installed.
Tupaia is installed on a machine whose `/home/ubuntu/tupaia/package.json` exists and gives its `name` as `tupaia`, the name of the Tupaia monorepo root.

> [!NOTE]
> A machine with no applications can never be ranked, so its issues never reach an incident; reporting Tupaia is what lets a Tupaia host's machine checks alert.

## Identifying a subject

A machine is identified by the identity its agent enrolled with, which the agent mints once and keeps.
Expand Down Expand Up @@ -63,6 +69,12 @@ A check reading an application's tables is about that application; a check gradi

An mSupply application carries two checks: the Canopy certificate collection check ([CHK-CCO](../canopy/certificate-collection-check.md)) and the Caddy certificate check ([CHK-CCT](caddy-certs.md)), each grading the DNS names belonging to it ([NAM](../canopy/names.md#which-application-a-dns-name-belongs-to)).

A Tupaia application carries no checks of its own.
Its machine carries the machine checks as any other machine does, and no Tamanu or Postgres check runs for it.

> [!NOTE]
> Tupaia is fronted by a web server other than Caddy and uses a database on another machine, so neither the certificate checks nor the database checks have anything of Tupaia's to grade.

A concern that genuinely exists on both sides is two checks rather than one check with a conditional subject, so neither has a mode in which it reports the wrong subject's reading.

Which checks apply to an application follows from its type, so a check written for one type is not run against another.
Expand All @@ -88,6 +100,9 @@ An application reports: its product version, its type, its install root where it

An mSupply application reports its type and its product version, read from the version its installation pins.

A Tupaia application reports its type and, as its product version, the full hash of the commit its checkout has checked out.
Its product version is absent when that commit cannot be determined.

A Postgres application reports its server version.
That version belongs to the server rather than to what connects to it, so an application using a database does not report the database's version as one of its own facts.

Expand Down
1 change: 1 addition & 0 deletions crates/alertd/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ pub mod store;
pub mod subject;
pub mod sweep;
pub mod sweep_cache;
pub mod tupaia;

pub use runtime::{CertificateSource, Compute, Duty, HttpRuntime, ServiceRuntime, TamanuDuty};
pub use stat::{MetricsSnapshot, Stat, StatKind, StatusCounts};
Expand Down
15 changes: 15 additions & 0 deletions crates/alertd/src/server_info.rs
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,21 @@ pub struct MsupplyInfo {
pub msupply_version: Option<String>,
}

/// The Tupaia installation's own facts.
///
/// Its type travels with the application report rather than here, and the
/// checked-out commit, as its product version, is the only other thing it
/// reports.
///
/// spec: SUBJ
#[derive(Debug, Clone, Default, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct TupaiaInfo {
/// The full hash of the checked-out commit, absent when it cannot be read.
#[serde(skip_serializing_if = "Option::is_none")]
pub tupaia_version: Option<String>,
}

/// Optional inputs sourced from the Tamanu DB / config that aren't trivially
/// available at gather time. Doctor populates these from its own DB connection.
#[derive(Debug, Clone, Default)]
Expand Down
40 changes: 38 additions & 2 deletions crates/alertd/src/subject.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use bestool_tamanu::ApiServerKind;
///
/// The wire type is an open set, so this enumerates only what bestool itself
/// reports from a host: its Tamanu deployment, the Postgres installation under
/// it, and mSupply. A machine commonly has Tamanu and Postgres, and they are
/// it, mSupply, and Tupaia. A machine commonly has Tamanu and Postgres, and they are
/// reported separately — "Tamanu as seen through its database" and "the health
/// of Postgres itself" are different questions about different things.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
Expand All @@ -25,16 +25,20 @@ pub enum ApplicationKind {
/// The mSupply installation on this machine, reported whether or not the
/// host has a Tamanu or a database.
Msupply,
/// The Tupaia installation on this machine, which carries no checks of its
/// own but is reported so the machine has an application to be ranked by.
Tupaia,
}

impl ApplicationKind {
/// Every kind bestool can report, for lookups that go from a wire key back
/// to the kind that produced it.
pub const ALL: [Self; 4] = [
pub const ALL: [Self; 5] = [
Self::TamanuCentral,
Self::TamanuFacility,
Self::Postgres,
Self::Msupply,
Self::Tupaia,
];

/// The application type as canopy names it.
Expand All @@ -44,6 +48,7 @@ impl ApplicationKind {
Self::TamanuFacility => "tamanu-facility",
Self::Postgres => "postgres",
Self::Msupply => crate::msupply::TYPE_SLUG,
Self::Tupaia => crate::tupaia::TYPE_SLUG,
}
}

Expand Down Expand Up @@ -89,6 +94,15 @@ impl ApplicationRef {
}
}

/// The Tupaia installed on the machine, of which there is one.
pub fn tupaia() -> Self {
let kind = ApplicationKind::Tupaia;
Self {
kind,
key: format!("host-{}", kind.type_slug()),
}
}

/// A Postgres cluster running on this machine, identified by the port it
/// answers on.
///
Expand Down Expand Up @@ -281,6 +295,7 @@ mod tests {
match kind {
ApplicationKind::Postgres => ApplicationRef::local_postgres(5432),
ApplicationKind::Msupply => ApplicationRef::msupply(),
ApplicationKind::Tupaia => ApplicationRef::tupaia(),
other => ApplicationRef::tamanu(other),
}
}
Expand Down Expand Up @@ -374,6 +389,27 @@ mod tests {
assert_eq!(msupply.kind.type_slug(), "msupply");
}

#[test]
fn no_scope_admits_tupaia() {
let tupaia = app_ref(ApplicationKind::Tupaia);
for scope in [
TamanuScope::Any,
TamanuScope::Central,
TamanuScope::Facility,
] {
assert!(!scope.admits(&tupaia), "{scope:?} admitted Tupaia");
}
assert!(!HostedScope::Any.admits(&tupaia));
assert!(!ApplicationKind::Tupaia.is_tamanu());
}

#[test]
fn tupaia_is_keyed_by_its_type() {
let tupaia = ApplicationRef::tupaia();
assert_eq!(tupaia.key, "host-tupaia");
assert_eq!(tupaia.kind.type_slug(), "tupaia");
}

#[test]
fn qualified_names_use_the_type_not_the_key() {
// Selection names a check on a kind of subject, so one invocation reaches
Expand Down
100 changes: 91 additions & 9 deletions crates/alertd/src/sweep.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ use crate::{
store,
subject::{ApplicationKind, ApplicationRef, Subject},
sweep_cache::SweepCache,
tupaia,
};

/// The name bestool's daemon reports under.
Expand Down Expand Up @@ -563,14 +564,15 @@ fn host_applications(
/// A Tamanu deployment is one; the Postgres under it is another, whether a
/// Tamanu uses it or the host has nothing but a `DATABASE_URL`. A cluster
/// reached at a remote address is still reported, keyed apart so no key claims
/// this machine hosts it. mSupply stands apart from both: it is reported
/// wherever it is installed, with no database and no targets at all.
/// this machine hosts it. mSupply and Tupaia stand apart from both: each is
/// reported wherever it is installed, with no database and no targets at all.
///
/// spec: SUBJ
fn sweep_applications(
targets: Option<&SweepTargets>,
tamanu_kind: Option<ApiServerKind>,
msupply_installed: bool,
tupaia_installed: bool,
) -> Vec<ApplicationRef> {
let mut applications = Vec::new();
if let Some(targets) = targets {
Expand All @@ -582,6 +584,9 @@ fn sweep_applications(
if msupply_installed {
applications.push(ApplicationRef::msupply());
}
if tupaia_installed {
applications.push(ApplicationRef::tupaia());
}
applications
}

Expand Down Expand Up @@ -933,6 +938,7 @@ pub async fn perform_sweep(
targets.as_ref(),
tamanu.as_ref().map(|t| t.kind),
msupply_installed,
tupaia::installed(),
);

// The machine's own context. It carries which Tamanu is installed here —
Expand Down Expand Up @@ -960,7 +966,8 @@ pub async fn perform_sweep(
//
// A cluster and a deployment exist only where the sweep resolved targets, so
// there is nothing to build their contexts from without them. mSupply needs
// none: it has no database and no supervised services.
// none: it has no database and no supervised services. Tupaia needs none
// either, carrying no checks of its own.
//
// One cache per sweep, so the readings that are the machine's — Canopy's
// entitlement, Caddy's configuration, the collected chains — are taken once
Expand Down Expand Up @@ -997,6 +1004,7 @@ pub async fn perform_sweep(
},
);
}
(ApplicationKind::Tupaia, _) => {}
// `sweep_applications` yields these only alongside targets.
(
ApplicationKind::Postgres
Expand Down Expand Up @@ -1153,6 +1161,9 @@ pub async fn perform_sweep(
ApplicationKind::Msupply => serde_json::to_value(&server_info::MsupplyInfo {
msupply_version: msupply::version(),
}),
ApplicationKind::Tupaia => serde_json::to_value(&server_info::TupaiaInfo {
tupaia_version: tupaia::version(),
}),
};
Ok((app.clone(), info.into_diagnostic()?))
})
Expand Down Expand Up @@ -2377,16 +2388,17 @@ mod tests {
#[test]
fn msupply_is_an_application_even_with_no_targets() {
assert_eq!(
sweep_applications(None, None, true),
sweep_applications(None, None, true, false),
vec![ApplicationRef::msupply()]
);
assert!(sweep_applications(None, None, false).is_empty());
assert!(sweep_applications(None, None, false, false).is_empty());
}

#[test]
fn a_tamanu_only_host_does_not_report_msupply() {
let targets = targets_for("postgresql://u@localhost/tamanu", true);
let applications = sweep_applications(Some(&targets), Some(ApiServerKind::Central), false);
let applications =
sweep_applications(Some(&targets), Some(ApiServerKind::Central), false, false);
assert_eq!(
applications,
vec![
Expand All @@ -2399,7 +2411,8 @@ mod tests {
#[test]
fn msupply_sits_beside_tamanu_and_postgres() {
let targets = targets_for("postgresql://u@localhost/tamanu", true);
let applications = sweep_applications(Some(&targets), Some(ApiServerKind::Facility), true);
let applications =
sweep_applications(Some(&targets), Some(ApiServerKind::Facility), true, false);
assert_eq!(
applications,
vec![
Expand All @@ -2410,7 +2423,7 @@ mod tests {
);

let generic = targets_for("postgresql://u@localhost/other", false);
let applications = sweep_applications(Some(&generic), None, true);
let applications = sweep_applications(Some(&generic), None, true, false);
assert_eq!(
applications,
vec![
Expand All @@ -2426,7 +2439,7 @@ mod tests {
/// spec: SUBJ
#[test]
fn an_msupply_host_runs_exactly_the_two_certificate_checks() {
let applications = sweep_applications(None, None, true);
let applications = sweep_applications(None, None, true, false);
let mut names = Vec::new();
for entry in checks::all() {
for dispatch in dispatches_for(&entry.run, &applications) {
Expand Down Expand Up @@ -2508,6 +2521,75 @@ mod tests {
assert_eq!(info, serde_json::json!({}));
}

#[test]
fn tupaia_is_an_application_even_with_no_targets() {
assert_eq!(
sweep_applications(None, None, false, true),
vec![ApplicationRef::tupaia()]
);
let targets = targets_for("postgresql://u@localhost/tamanu", true);
assert!(
!sweep_applications(Some(&targets), Some(ApiServerKind::Central), true, false)
.contains(&ApplicationRef::tupaia())
);
}

/// On a host with Tupaia and nothing else, no application check runs, while
/// every machine check still does.
///
/// spec: SUBJ
#[test]
fn a_tupaia_host_runs_only_machine_checks() {
let applications = sweep_applications(None, None, false, true);
let registry = checks::all();
let mut machine_checks = 0;
for entry in &registry {
for dispatch in dispatches_for(&entry.run, &applications) {
match dispatch.subject() {
Subject::Machine => machine_checks += 1,
Subject::Application(app) => {
panic!("{} ran for {app:?}", entry.name)
}
}
}
}
let expected = registry
.iter()
.filter(|entry| matches!(entry.run, checks::Run::Machine(_)))
.count();
assert_eq!(machine_checks, expected);
assert!(machine_checks > 0);
}

/// A Tupaia application reports its type and its checked-out commit, and
/// none of another application's facts.
///
/// spec: SUBJ
#[test]
fn tupaia_reports_its_type_and_version_only() {
let commit = "0123456789abcdef0123456789abcdef01234567";
let details = vec![(
ApplicationRef::tupaia(),
serde_json::to_value(server_info::TupaiaInfo {
tupaia_version: Some(commit.into()),
})
.unwrap(),
)];
let payload = build_payload(&machine_info(), &details, &[]).unwrap();
let report = payload
.applications
.as_ref()
.unwrap()
.get("host-tupaia")
.unwrap();
assert_eq!(report.type_, "tupaia");
assert_eq!(report.detail.len(), 1);
assert_eq!(report.detail.get("tupaiaVersion").unwrap(), commit);

let info = serde_json::to_value(server_info::TupaiaInfo::default()).unwrap();
assert_eq!(info, serde_json::json!({}));
}

/// A sweep that observes an application's compute to be off drops the state
/// that does not outlive it, and keeps the state that does. Doing it here
/// rather than in each check is what makes it impossible to forget.
Expand Down
Loading
Loading