feat: Structured audit layer middleware

[HatemMn]

About merging

To keep everything consistent, the following PRs should be rebased on top of each other in this order

1. feat: Structured audit layer middleware #934
2. fix: no usernames in Otel #1063
3. Database PR: coming soon
4. Siem PR: later

---

Specs (optional) :

• PCI-DSS compliance : important, won't drop it ( source : check PCI-DSS v4.0.1 Req. 10 )
• HIPPA Compliance : ( Source : https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-66r2.pdf - section 5.3.2. Audit Controls (§ 164.312(b)) )

---

Artifacts :

For testing purposes, these debug artifacts are shared :

no_auth_audit.toml

default_username = "admin"

[http]
port = 9998
hostname = "0.0.0.0"

[db]
database_type = "sqlite"
sqlite_path = "/tmp/kms-data-e2e"
clear_database = true
unwrapped_cache_max_age = 15

[logging]
rust_log = "info,cosmian_kms=info"
ansi_colors = false

[ui_config]
ui_index_html_folder = "ui/dist"

[audit]
enabled = true

[audit.file]
path = "/tmp/kms-audit-e2e.jsonl"

The following commands (all assume you already ran a KMS) :

# Terminal 2 (while server is running)
cargo run -p ckms --features non-fips -- --url http://127.0.0.1:9998 sym keys create --algorithm aes --number-of-bits 256 --tag audit-e2e-key
cargo run -p ckms --features non-fips -- --url http://127.0.0.1:9998 sym encrypt -k d5d986ac-ff6c-472c-904e-0b0bf6bfaa92 /tmp/audit-plain.txt
curl -s -o /tmp/kmip-empty.json -w "KMIP_EMPTY_HTTP=%{http_code}\n" -X POST -H "Content-Type: application/json" -d '{}' http://127.0.0.1:9998/kmip/2_1

cargo run -p ckms --features non-fips -- --url http://127.0.0.1:9998 sym keys create --algorithm aes --number-of-bits 256 --tag audit-e2e-key-r2
cargo run -p ckms --features non-fips -- --url http://127.0.0.1:9998 sym encrypt -k 417fe2de-827d-48d0-8d51-851bec315b76 /tmp/audit-plain.txt
cargo run -p ckms --features non-fips -- --url http://127.0.0.1:9998 sym decrypt -k 417fe2de-827d-48d0-8d51-851bec315b76 /tmp/audit-plain.enc
cargo run -p ckms --features non-fips -- --url http://127.0.0.1:9998 sym decrypt -k 00000000-0000-0000-0000-000000000000 /tmp/audit-plain.enc
curl -s -o /tmp/kmip-empty-2.json -w "KMIP_EMPTY_HTTP=%{http_code}\n" -X POST -H "Content-Type: application/json" -d '{}' http://127.0.0.1:9998/kmip/2_1

Normally produce kms-audit.jsonl :

{"id":0,"timestamp":"2026-07-20T19:02:25.605113314Z","operation":"Create","user":"admin","object_uid":"359019d8-1543-4e2e-9d96-674dd64fcffc","algorithm":"AES","client_ip":"127.0.0.1","result":"Success","duration_ms":4,"request_id":"4b2e6994-0ef9-4dfe-9804-0da4c5edde4b","prev_hash":"0000000000000000000000000000000000000000000000000000000000000000","row_hash":"641c442edc2bb075e9770a905c843fdadfb85503464e67c1658549624d44fb00"}
{"id":1,"timestamp":"2026-07-20T19:02:36.704410987Z","operation":"Encrypt","user":"admin","object_uid":"359019d8-1543-4e2e-9d96-674dd64fcffc","algorithm":"AES","client_ip":"127.0.0.1","result":"Success","duration_ms":12,"request_id":"3618ade8-5db7-4635-9d05-5af6a7614d52","prev_hash":"641c442edc2bb075e9770a905c843fdadfb85503464e67c1658549624d44fb00","row_hash":"637cd569144c90f0ee73564286f8a75c56084cf76f7375b3d31266161b3de6a3"}
{"id":2,"timestamp":"2026-07-20T19:02:43.474676849Z","operation":"kmip","user":"admin","object_uid":null,"algorithm":null,"client_ip":"127.0.0.1","result":{"Failure":"422 Unprocessable Entity"},"duration_ms":0,"request_id":"029b157a-4d66-4aa8-a504-dab433baa374","prev_hash":"637cd569144c90f0ee73564286f8a75c56084cf76f7375b3d31266161b3de6a3","row_hash":"3b00ac7a2880cfe79c30763b3e6e18c37a8019dbe1687d6a1e4936ddd4772695"}
{"id":3,"timestamp":"2026-07-20T19:02:50.496997982Z","operation":"Create","user":"admin","object_uid":"021c29e9-8017-42d6-9887-bebb5c413e74","algorithm":"AES","client_ip":"127.0.0.1","result":"Success","duration_ms":2,"request_id":"2e5fcc19-2021-45a3-822c-9ecf4e38de85","prev_hash":"3b00ac7a2880cfe79c30763b3e6e18c37a8019dbe1687d6a1e4936ddd4772695","row_hash":"b75759b556141c03e2e4c9b6cfb4e8992f3de09338bb32b46eb549b0e1a6f30c"}
{"id":4,"timestamp":"2026-07-20T19:03:00.273344336Z","operation":"Encrypt","user":"admin","object_uid":"021c29e9-8017-42d6-9887-bebb5c413e74","algorithm":"AES","client_ip":"127.0.0.1","result":"Success","duration_ms":2,"request_id":"330d8643-91a4-4869-af8c-849ee50d5643","prev_hash":"b75759b556141c03e2e4c9b6cfb4e8992f3de09338bb32b46eb549b0e1a6f30c","row_hash":"c849c7efd68569ced3b63af96af2ad5d77a82051e36c64c2ef9f5be7eeea17e9"}
{"id":5,"timestamp":"2026-07-20T19:03:00.685874574Z","operation":"Decrypt","user":"admin","object_uid":"021c29e9-8017-42d6-9887-bebb5c413e74","algorithm":"AES","client_ip":"127.0.0.1","result":"Success","duration_ms":0,"request_id":"29df4906-b8c2-47fb-b88b-b8ca3a0c6ceb","prev_hash":"c849c7efd68569ced3b63af96af2ad5d77a82051e36c64c2ef9f5be7eeea17e9","row_hash":"fbca602d562d8205315085ddf9a5493eee14be73180ff79c023061027ea6cde9"}
{"id":6,"timestamp":"2026-07-20T19:03:01.069844169Z","operation":"Decrypt","user":"admin","object_uid":"00000000-0000-0000-0000-000000000000","algorithm":"AES","client_ip":"127.0.0.1","result":{"Failure":"422 Unprocessable Entity"},"duration_ms":1,"request_id":"7a297435-64f3-41ef-a0b2-df64146915de","prev_hash":"fbca602d562d8205315085ddf9a5493eee14be73180ff79c023061027ea6cde9","row_hash":"ad83e2e8730ed5556bf51f2d8b301167cbe54b1314a2a1ce720a274b35f84594"}
{"id":7,"timestamp":"2026-07-20T19:03:01.079329139Z","operation":"kmip","user":"admin","object_uid":null,"algorithm":null,"client_ip":"127.0.0.1","result":{"Failure":"422 Unprocessable Entity"},"duration_ms":0,"request_id":"236a3050-940d-4005-8cbd-f25a4a33627a","prev_hash":"ad83e2e8730ed5556bf51f2d8b301167cbe54b1314a2a1ce720a274b35f84594","row_hash":"a361c07f62b719755b3e5ff687049db728e0c1cdb722a8e3060369f17dd01610"}

Then you can run :

cargo run -p ckms --features non-fips -- audit verify --path #your file

Other technical considerations

Reverse proxy usage :

    /// Comma-separated list of trusted reverse-proxy IP addresses or CIDR blocks
    /// (e.g. `"10.0.0.0/8,172.16.0.0/12"`).  The `X-Forwarded-For` header is only
    /// trusted when the actual TCP peer address falls within one of these ranges.
    ///
    /// When empty (the default), `client_ip` is always the direct TCP peer address.
    /// Single IPs can be expressed as `/32` (IPv4) or `/128` (IPv6).
    #[clap(
        long = "audit-trusted-proxy-cidrs",
        env = "KMS_AUDIT_TRUSTED_PROXY_CIDRS",
        value_delimiter = ',',
        verbatim_doc_comment
    )]
    #[serde(rename = "trusted_proxy_cidrs", default)]
    pub audit_trusted_proxy_cidrs: Vec<IpNet>,

This parameter seems over engibneered, but :

Threat without allowlist, two bad options:

Trust XFF always → any attacker sends fake X-Forwarded-For: 1.2.3.4 header directly to KMS → audit log records wrong [client_ip]. Forensic/compliance trail poisoned — attacker hides real origin or frames innocent IP. This is audit-log spoofing (CWE-117-ish), works via any protocol (UI, CLI, curl), no UI needed.
Never trust XFF → when real proxy present, [client_ip] always = proxy's own IP → useless for forensics regardless of caller.
CIDR allowlist fixes both: trust XFF only from known proxy IPs, so external attacker can't inject fake header, but real proxy's forwarded value still recorded.

---

What's audited, what's not

Audited

• POST/GET /kmip — kmip::kmip_2_1_json (JSON) and kmip::kmip (binary TTLV)
• access::* — grant/revoke/list access rights, list owned objects, get current user
• get_server_info, get_hsm_status, /download-cli

Not audited

• Raw TCP socket server — start_socket_server → handle_ttlv_bytes (start_kms_server.rs). Full KMIP protocol, no AuditMiddleware possible (not Actix), no manual audit call either.
• /v1/crypto REST Native Crypto API (jose/) — POST /encrypt, /decrypt, /sign, /verify, /mac, POST /keys (create), DELETE /keys/{kid} (destroy), POST /keys/unwrap, tag add/remove/list.
• Vault-compatible Transit + PKI API — /v1/{transit_mount} (create_transit_key, configure_transit_key, delete_transit_key, sign_with_transit_key, ...) and /v1/{pki_mount} (sign_intermediate), gated by vault_api_enabled. /v1/auth proxy scope also unaudited (pure forward, no key ops).
• /tokenize (non-fips only) — hash, noise, word_mask, word_tokenize, word_pattern_mask, aggregate_number, aggregate_date, scale_number.
• /google_cse, /ms_dke, /aws (XKS), /azureekm — already decided (SEC-04, ejected).

Not audited, but by design (no key operations)

• ui — SPA static files + login
• /.well-known/jwks.json — public key export
• health, get_version, swagger/OpenAPI routes, root_redirect

[HatemMn]

Note: needs 2 reviewers

[Manuthor]

Can you justify the choice of the JSONL format for the audit file?

[HatemMn]

To put it simply:

• JsonL is one of the most cummonly use formats for logs
• the "L" (one entry by line) avoids entirely every issue with formatters and such, while keeping the nested nature of Json that extractors need
• No overhead, No boilerplates, Just logs and some delimiters
• breakages (exp: incomplete line) can be trivially detected
• already ingested by Splunk or even a simple grep for indicident response. Also Given that Splunk-UF-on-raw-JSONL is the only production export path we currently have, storing JSONL means the forwarder reads the authoritative file directly. No transform, no second copy to keep in sync. Any other format will oblige us to either disable splunk export and forward it to a new PR or add a synchro macanism that will make the system slower and error prone

For the syslog question of the screenshot, yes the standard exists, but it's dead, and the living implementation is proprietary and doesn't do what we'd be doing anyway.

[Manuthor]

And you do confirm that JSONL format is consumable by Splunk, Datadog, etc.?

[HatemMn]

And you do confirm that JSONL format is consumable by Splunk, Datadog, etc.?

To my understanding there is a problem with this question, please correct me if I am mixing things:our local file where we store data and "what we push to external systems" are two separate matters.

First matter

The first is what this PR is about and is our decision that is have justified by these arguments above

Second matter

The second point is to be done here: #937

Storing our data in JsonL does not have inference to how we will export it to Splunk or friends, here is a doc example :

kms/documentation/docs/kms_clients/audit.md

Line 81 in e7bb21c

Export events since a given date as CEF, piped to a SIEM:

Currently, an non live export to those guys is technically possible

Thanks to a small hack I have documented, we can technically start live exporting now, but that's not yet a feature of the KMS. It's rather a hack smart operators can pull out - and that I preferred to document 'to expect loss' rather than wait for people to do it and get errors (we can control the KMS, but not what users do with their terminal)

[HatemMn]

Sorry for the late posting, I forgot to post this last week

[HatemMn]

To put it simply:

• JsonL is one of the most cummonly use formats for logs
• the "L" (one entry by line) avoids entirely every issue with formatters and such, while keeping the nested nature of Json that extractors need
• No overhead, No boilerplates, Just logs and some delimiters
• breakages (exp: incomplete line) can be trivially detected
• already ingested by Splunk or even a simple grep for indicident response. Also Given that Splunk-UF-on-raw-JSONL is the only production export path we currently have, storing JSONL means the forwarder reads the authoritative file directly. No transform, no second copy to keep in sync. Any other format will oblige us to either disable splunk export and forward it to a new PR or add a synchro macanism that will make the system slower and error prone

For the syslog question of the screenshot, yes the standard exists, but it's dead, and the living implementation is proprietary and doesn't do what we'd be doing anyway.