Skip to content

test(ci): add JSONL audit compat checks for OpenSearch and Splunk - #1131

Open
chokoblitz wants to merge 3 commits into
developfrom
ci/audit-jsonl-compat
Open

test(ci): add JSONL audit compat checks for OpenSearch and Splunk#1131
chokoblitz wants to merge 3 commits into
developfrom
ci/audit-jsonl-compat

Conversation

@chokoblitz

Copy link
Copy Markdown
Contributor

What

Two ephemeral (--rm, no volume) CI checks validating that the KMS JSONL audit
format ingests cleanly into OpenSearch (bulk API, strict mapping) and Splunk
(HEC), with a shared Python validation script and versioned schema files
(schemas/kms-audit-*.json). Follow-up to #934.

Finding

result field is polymorphic ("Success" | {"Failure": "..."}) and silently
drops from Splunk's default _json field extraction when it's an object. Both
backends now get a normalized result_status/result_detail pair instead, so
the comparison is apples-to-apples. Worth discussing upstream whether the audit
schema itself should avoid polymorphic fields — not blocking here.

Security note — CI-only relaxations

Both backends run with security features disabled, strictly scoped to the
ephemeral test container
, never touching real KMS data or a production system:

  • OpenSearch: DISABLE_SECURITY_PLUGIN=true — auth/TLS off, container is
    --rm, no volume, torn down at job end.
  • Splunk: self-signed TLS cert (default for a fresh container) — validation
    script skips cert verification only when --insecure is explicitly passed by
    the task, never a script default.

Testing

Both pass locally (mise run test:audit-compat-{opensearch,splunk}, 8/8 events
each). Images pinned (OpenSearch 3.8.0, Splunk 10.4.2), not :latest
reproducibility over drift, per team discussion.

Unrelated fixes bundled (pre-existing, blocked all commits on develop)

  • deny.toml: ignore RUSTSEC-2026-0258 (h2, low severity, real upgrade tracked separately)
  • lychee.toml: exclude sekretz.com (dead placeholder domain in deny.toml's own template comment)

@chokoblitz
chokoblitz requested a review from Manuthor August 20, 2026 12:22
@HatemMn
HatemMn self-requested a review August 26, 2026 16:16
@HatemMn HatemMn self-assigned this Aug 26, 2026

@HatemMn HatemMn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chokoblitz thanks for the PR but in the current state something needs to be changed or it should be closed and kept as artifact/proof (i'll explain)

the two workflows test against a static fixture. The fixture was indeed been generated by the KMS, but if later on we change something in the KMS, those tests won't catch it, so there is no goal of the CI (for now)

Worse,

Images pinned (OpenSearch 3.8.0, Splunk 10.4.2), not :latest —
reproducibility over drift, per team discussio

Since they are pinned, when can't even (empiracally) discover a breaking change that Splunk or OpenSearch produce and makes them incompatible with our system (it's actually the one use case where a static fixture might be an acceptable idea)

Anyway, two paths:

  • Save and archive this PR, save it to the internal docs (Notion) with any other useful info like configuring opensearch & splunk
  • Make that file sample-audit.jsonL generated by a live KMS: that's the path I would prefer we take

feel free to give me your opinion on the matter

@HatemMn HatemMn assigned chokoblitz and unassigned HatemMn Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants