Go SDK 6/8: prefix privacy - #517
Open
bkeroack wants to merge 2 commits into
Open
Conversation
A prefix watch trades precision for privacy: the consumer registers a bits-wide prefix of sha256(scriptPubKey) and the node delivers every transaction falling in that 2^-bits bucket, so it learns the bucket and never the script. The cost is decoys, which the consumer has to filter out locally -- and that filter has to be local, because any precise follow-up fetch would re-leak the exact script the bucket was hiding. PrefixWatcher is that filter. It holds the real scriptPubKeys, and for each delivery it recomputes sha256(scriptPubKey) over every output and every retained spent prevout, reporting only the true hits. Prevouts the node did not retain the script for cannot be settled locally, so they surface separately as Unresolved rather than being folded into either answer; treating them as non-matches would silently drop real spends. Two things differ from a naive port of the Rust SDK. Prefixes are masked below the declared width. The wire carries whole bytes, so a 12-bit bucket ships 2 bytes, and the node keys on the top 12 bits only. Shipping the remaining 4 bits unmasked would hand over more scripthash than the declared bucket width and narrow the anonymity set for free. Masking is free -- the node derives the identical bucket key either way -- and it is what makes narrow buckets actually collapse: at 1 bit, any number of scripts now registers at most two buckets rather than one per distinct leading byte. This is a deliberate divergence in wire output from the Rust SDK and PR 7's parity harness has to account for it. Filtering needs the transaction's outputs, its input outpoints, and its txid, which means a decoder. The SDK depends on no Bitcoin library by design, so tx.go supplies exactly those three things and nothing else. It accepts both serializations and always hashes the txid over the legacy one, which is what makes the id witness-invariant and what the node's wire txids are. The decoder is fed length prefixes it did not choose, so every count and length is checked against the bytes actually remaining before it sizes an allocation. That bound divides rather than multiplies: count * minBytesEach wraps uint64 for a count near 2^64 and would wave through the very payload it exists to reject. There is a test for exactly that wrap. Those allocation tests deliberately claim ~1e6 items and not ~4e9. The decoder rejects both identically, but the assertion only means anything when the bound is broken -- and a broken bound turns a 4-billion count into ~137 GB resident, which does not fail an 8 MB budget assertion, it OOMs the host and takes the job's cgroup with it. A million items is just as impossible for a 50-byte payload and costs 37 MB to prove. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HZw7Sf8PZj3qPJ6orxDH17
The test waited for the loader's third attempt and then asserted on `srv.legCount()-1`, on the assumption that a third attempt implies a third connection. It does not: `attempts` is incremented on ENTRY to the loader, so it reaches 3 before the leg that attempt belongs to has been registered server-side. The assertion then interrogates a still-empty leg and fails with "leg 1 recorded 0 of 1 control messages". Deriving the index from a counter that races the reconnect was the bug. Poll every leg for the AddScripts control instead, which is what the test actually means: the eventually-loaded set is registered somewhere, on whichever connection finally succeeded. Caught by CI on #517 -- a PR that touches no Go runtime code -- which is the flake presenting as an unrelated failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HZw7Sf8PZj3qPJ6orxDH17
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sixth of eight PRs implementing the Go SDK per
SATD_GO_SDK_PLAN.md.Stack — merge bottom-up. Base is #516 (
feat/go-sdk-resilient-watch), which is itself based on #515.What this adds
PrefixWatcher, the local re-filter that makes prefix watching usable, plus the minimal transaction decoder it needs.A prefix watch registers a
bits-wide prefix ofsha256(scriptPubKey); the node delivers every transaction in that2^-bitsbucket, so it learns the bucket and never the script. The consumer then has to filter the decoys out locally — any precise follow-up fetch would re-leak the exact script the bucket was hiding.PrefixWatcherholds the real scriptPubKeys and, per delivery, recomputessha256(scriptPubKey)over every output and every retained spent prevout, returning only genuine hits.Prevouts whose script the node did not retain (a mempool spend below the
fullretention tier) cannot be settled locally. Those surface asPrefixHits.Unresolvedrather than being folded into either answer — reporting them as non-matches would silently drop real spends.HasUnresolved()exists so callers cannot conclude "no match" by accident.Two deliberate divergences from a naive port
1. Prefixes are masked below their declared width. The wire carries whole bytes, so a 12-bit bucket ships 2 bytes, and the server keys on
(bits, top32 & prefix_mask(bits))— the top 12 bits only. Shipping the low 4 bits unmasked hands the node more scripthash than the declared bucket width and narrows the anonymity set for nothing in return. Masking costs nothing (identical bucket key server-side) and is what makes narrow buckets actually collapse: at 1 bit, any number of scripts registers at most two buckets instead of one per distinct leading byte.satd-events-client), which sends the unmasked bytes. PR 7's parity harness must account for it. Flagging it explicitly rather than letting the harness discover it as a spurious diff.2. An in-tree decoder. Filtering needs outputs, input outpoints, and the txid. The SDK depends on no Bitcoin library by design, so
tx.goprovides exactly those and nothing else. Both serializations are accepted; the txid is always hashed over the legacy one, which is what makes it witness-invariant and what the node's wire txids are.It is a decoder, not a validator — it is only ever fed bytes the node produced, so the risk it guards is a malformed or truncated payload, not consensus divergence.
Allocation bounds
The decoder is handed length prefixes it did not choose, so every count and length is checked against the bytes actually remaining before sizing an allocation.
The bound divides rather than multiplies:
count * minBytesEachwrapsuint64for a count near 2^64 and would wave through exactly the payload it exists to reject. There is a test pinning that — an input count chosen socount * 41wraps down to 4, precisely the bytes left.The allocation tests claim ~1e6 items rather than ~4e9 on purpose. The decoder rejects both identically, but the assertion only matters when the bound is broken, and a broken bound turns a 4-billion count into ~137 GB resident. That does not fail an 8 MB budget assertion — it OOMs the machine and takes the job's whole cgroup down with it. This was not hypothetical: it killed a dev box twice while validating this PR. A million items is just as impossible for a 50-byte payload and costs 37 MB to prove.
Testing
./lint.sh(fmt/vet/staticcheck/errcheck) clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01HZw7Sf8PZj3qPJ6orxDH17