Skip to content

fix(lens): grade EL rebuilds against the upstream benchmark - #803

Open
remyluslosius wants to merge 1 commit into
mainfrom
fix/el-rebuild-compliance-lens
Open

fix(lens): grade EL rebuilds against the upstream benchmark#803
remyluslosius wants to merge 1 commit into
mainfrom
fix/el-rebuild-compliance-lens

Conversation

@remyluslosius

Copy link
Copy Markdown
Contributor

A Rocky, AlmaLinux, Oracle Linux or CentOS host is offered no STIG or CIS lens at all.

Confirmed on a live host rather than from source. owas-rock09 (almalinux 9.8) had 769 rule results stored, 391 of them carrying a stig_rhel9 ref, and the lens picker showed only "All rules" and "NIST 800-53".

owas-tst02 (rhel 9.6) owas-rock09 (almalinux 9.8)
Lenses offered All rules · CIS RHEL 9 · NIST 800-53 · STIG RHEL 9 All rules · NIST 800-53
STIG 88.3%, 401 rules not offered
CIS 79.2%, 313 rules not offered

Cause

Two paths derived the lens key from the raw distro id:

  • OSResolvedMatchSQL built <family>_<lower(os_family)><major>, asking for stig_almalinux9. No rule in the corpus emits that; Kensa authors OS keys for rhel8/9/10 and ubuntu22/24 only.
  • frameworkCompatibleWithOS rejected stig_rhel9 because the pinned family rhel did not equal the host family almalinux.

The second is why this was invisible rather than merely wrong. It decides which lenses are offered, so the symptom was not a zero score but an absent lens: a page that looks complete while withholding two thirds of the host's compliance signal, with nothing on screen to say so.

The product already had the right mapping, in the wrong layer. osLabel.ts mapped almalinux → RHEL for display, so the header read "RHEL 9.8" while the query asked for almalinux.

Fix

Two independent copies of one fact is how this shipped, so: one framework.BenchmarkFamily consumed by both paths, with the SQL CASE generated from the Go map rather than hand-mirrored. AC-13 asserts both paths consume it, not that they happen to agree today.

Fedora and Amazon Linux are deliberately excluded. Fedora rolls up to the rhel family in Discovery, so the obvious rollup-based fix would have swept it in and graded a Fedora host against the RHEL 9 STIG, manufacturing coverage rather than reporting none. A distro absent from the map keeps its own id and is offered no OS-specific lens. The negative cases carry the AC.

osLabel also gains fedora as its own label instead of falling through to "Unknown" (which read as a detection failure that had not happened), plus ol/oracle in the RHEL display bucket. The table now carries a comment that it is a display mapping and not a benchmark-equivalence claim, since copying it backend-side is exactly what would reintroduce this.

Why nothing caught it

Every host fixture in the tree sets os_family to rhel or ubuntu.

Upstream

The normalization is a stand-in for a fact Kensa computes and discards: it resolved the host to an EL platform in order to decide which rules to run, and does not publish what it resolved to. Filed as features/KN-OW-018. Delete this map when that lands rather than extending it, since every entry is a second copy of Kensa's platform gate and the gate is expected to widen.

Worth noting the stored data cannot answer it either: framework_refs is a property of the rule, so an AlmaLinux host and a RHEL host come back carrying the same five STIG keys, including two Ubuntu ones.

Verification

  • system-compliance-lens 1.4.0 → 1.5.0 (C-07, AC-12, AC-13); frontend-host-list-os 1.0.0 → 1.1.0 (AC-08)
  • 118/118 specs at 100% annotation coverage; Go tests and 387 frontend tests pass; gofmt clean
  • Mutation-checked all three guards: reverting the SQL to lower(), adding fedora to the map, and reverting the lens comparison each fail loudly
  • Verified live after the fix: the same host now reports STIG RHEL 9 at 41.4% over 391 rules and CIS RHEL 9 at 49.2% over 303, and the Compliance tab regained its failing-count badge

Blast radius

Every Rocky, AlmaLinux, Oracle Linux and CentOS Stream host. Those platforms are named in the CMMC Sprint scope, and drift detection is computed from the compliance score that was coming back empty.

A Rocky, AlmaLinux, Oracle Linux or CentOS host was offered no STIG or CIS
lens at all. Confirmed on a live host: owas-rock09 (almalinux 9.8) had 769
rule results stored, 391 of them carrying a stig_rhel9 ref, and the picker
showed only "All rules" and "NIST 800-53".

Two paths derived the lens key from the raw distro id. OSResolvedMatchSQL
built <family>_<lower(os_family)><major>, asking for stig_almalinux9, which
no rule in the corpus emits; Kensa authors OS keys for rhel8/9/10 and
ubuntu22/24 only. frameworkCompatibleWithOS then rejected stig_rhel9 because
the pinned family "rhel" did not equal the host family "almalinux".

That second one is why this was invisible rather than merely wrong. It
decides which lenses are OFFERED, so the symptom was not a zero score but an
absent lens: a page that looks complete while withholding two thirds of the
host's compliance signal, with nothing on screen to say so.

The product already had the right mapping in the wrong layer. osLabel.ts
mapped almalinux to RHEL for display, so the header read "RHEL 9.8" while the
query asked for almalinux. Two independent copies of one fact is how this
shipped, so the fix is one BenchmarkFamily consumed by both paths, with the
SQL CASE generated from the Go map rather than hand-mirrored. AC-13 asserts
both paths consume it, not that they happen to agree.

Fedora and Amazon Linux are deliberately excluded. Fedora rolls up to the
rhel family in Discovery, so the obvious rollup-based fix would have swept it
in and graded a Fedora host against the RHEL 9 STIG, manufacturing coverage
rather than reporting none. A distro absent from the map keeps its own id and
is offered no OS-specific lens, which is the honest outcome for a platform
nobody has verified. The negative cases carry the AC.

osLabel gains fedora as its own label instead of falling through to
"Unknown", which read as a detection failure that had not happened, plus
ol/oracle in the RHEL display bucket. The table now carries a comment that it
is a display mapping and not a benchmark-equivalence claim, since copying it
backend-side is exactly the mistake that would reintroduce this.

Nothing caught it because every host fixture in the tree sets os_family to
rhel or ubuntu.

The normalization is a stand-in for a fact Kensa computes and discards: it
resolved the host to an EL platform to decide which rules to run, and does
not publish what it resolved to. CP features/KN-OW-018 asks for that. Delete
this map when it lands rather than extending it, since every entry is a
second copy of Kensa's platform gate and the gate is expected to widen.

system-compliance-lens 1.4.0 -> 1.5.0 (C-07, AC-12, AC-13).
frontend-host-list-os 1.0.0 -> 1.1.0 (AC-08). Mutation-checked on all three
guards. Verified live: the same host now reports STIG RHEL 9 at 41.4% over
391 rules and CIS RHEL 9 at 49.2% over 303.
@remyluslosius
remyluslosius force-pushed the fix/el-rebuild-compliance-lens branch from 5ff471b to f0adc2b Compare August 5, 2026 09:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant