feat(enc): encrypt list elements and stream field values - #35
Open
mattyhogan wants to merge 2 commits into
Open
Conversation
LPUSH/RPUSH and XADD accept a trailing ENCRYPTED flag (mirroring SET/HSET): elements/field-values are sealed as envelopes at rest and decrypted on read. List AAD is key-independent so LMOVE/RPOPLPUSH relocate ciphertext without re-keying; stream AAD binds key+field. Encrypted pushes self-log ENC RAWLPUSH/RAWRPUSH and XADD self-logs the resolved ciphertext, so WAL replay is deterministic. Also fixes a pre-existing bug: the shard-local read fast-path has no keyring and returned ciphertext for encrypted GET/HGET/LRANGE/XRANGE over RESP. Reads (and pipelines) now fall through to the decrypting slow path whenever encryption is active.
REWRAP only re-wrapped Str/StrBuf/Hash, so encrypted list elements and stream field values kept their DEK wrapped under the old key; after RETIRE they became undecryptable. REWRAP now rewraps List and Stream in place and the RETIRE guard checks them too. Both also persist and truncate the WAL so the rewrap is durable across a restart (a WAL replay was reverting in-memory rewraps to the pre-rewrap ciphertext).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends value-level encryption-at-rest (already supported for strings, hashes, and table columns) to the remaining opaque-value primitives: lists and streams.
API
LPUSH key v1 v2 ... ENCRYPTED/RPUSH ... ENCRYPTED— seal each element.XADD key [opts] id field val ... ENCRYPTED— seal each field value.Mirrors the existing
SET/HSET ... ENCRYPTEDconvention. Encryption is per-write; reads auto-detect the envelope (LUXENC2magic) and decrypt, so encrypted and plaintext entries coexist.Design
__lux_list/element), soLMOVE/RPOPLPUSHrelocate an encrypted element to another key without re-keying and it still decrypts. Stream AAD binds key + field.ENC RAWLPUSH/RAWRPUSH; encrypted XADD self-logs a resolved XADD carrying the ciphertext (flag stripped). Replay stores envelopes verbatim, no re-encryption.Bundled bug fix
The shard-local read fast-path reads stored bytes directly and has no keyring, so it returned ciphertext for encrypted
GET/HGET/LRANGE/XRANGEover RESP (pre-existing; shipped in v0.25.0, but latent — the in-crate tests exercise the slow executor, and table/HTTP encryption was unaffected). Reads and pipelines now fall through to the decrypting slow path whenever encryption is active. Caught by the live e2e, not the unit tests — hence the new TCP integration test.Limitations (documented)
ENC INIT.Tests
Unit: LPUSH/XADD WAL-ciphertext + replay round-trips, LMOVE-across-keys. Integration (
tests/encryption.rs): RESP-over-TCP round-trips for encrypted string/hash/list/stream + LMOVE. Live e2e confirmed plaintext round-trips, LMOVE portability, and no plaintext on disk. Full suite 909 pass, clippy clean.