Event feed connector: foundations (1/3) - #777
Conversation
There was a problem hiding this comment.
Pull request overview
Introduces the foundational Go event-feed components required by the forthcoming connector run loop.
Changes:
- Adds event models, seams, codecs, filtering, deduplication, timing, and checkpoint persistence.
- Adds a credential-isolated WebSocket transport and URL security policies.
- Adds deterministic test fakes and extensive contract/unit coverage.
Tip
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.
Reviewed changes
Copilot reviewed 41 out of 42 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
AGENTS.md |
Registers the sanctioned event-feed architecture. |
go/go.mod |
Adds the WebSocket dependency. |
go/go.sum |
Records dependency checksums. |
eventfeed/backoff.go |
Implements retry and repair jitter. |
eventfeed/backoff_test.go |
Tests timing boundaries and saturation. |
eventfeed/cable.go |
Implements Action Cable frame codecs. |
eventfeed/cable_test.go |
Tests frame parsing and commands. |
eventfeed/checkpoint.go |
Defines checkpoint identity and store seam. |
eventfeed/clock.go |
Provides the timer abstraction and system clock. |
eventfeed/clock_test.go |
Tests system timer registration. |
eventfeed/continuation.go |
Validates continuation origins. |
eventfeed/continuation_test.go |
Tests continuation security policy. |
eventfeed/dedupe.go |
Implements delivered-event LRU deduplication. |
eventfeed/dedupe_test.go |
Tests deduplication and eviction. |
eventfeed/digest.go |
Implements canonical filter digests. |
eventfeed/digest_test.go |
Verifies shared digest fixtures. |
eventfeed/doc.go |
Documents the package architecture. |
eventfeed/errors.go |
Defines terminal errors and reasons. |
eventfeed/errors_test.go |
Tests error taxonomy and rendering. |
eventfeed/event.go |
Defines event payloads. |
eventfeed/event_test.go |
Tests payload presence semantics. |
eventfeed/filestore.go |
Implements bounded atomic checkpoint storage. |
eventfeed/filestore_test.go |
Tests persistence, locking, and file safety. |
eventfeed/filters.go |
Defines and validates feed filters. |
eventfeed/filters_test.go |
Tests validation and cloning. |
eventfeed/redact.go |
Redacts observer-facing URLs. |
eventfeed/redact_test.go |
Tests credential-safe URL rendering. |
eventfeed/seams.go |
Defines connector interfaces and public types. |
eventfeed/transport.go |
Implements cable URL policy. |
eventfeed/transport_test.go |
Tests URL and proxy policy. |
eventfeed/transport_contract_test.go |
Defines the shared transport contract. |
eventfeed/websocket_transport.go |
Implements the default WebSocket transport. |
eventfeed/websocket_transport_test.go |
Tests real transport behavior and security. |
eventfeed/feedtest/clock.go |
Provides deterministic virtual time. |
eventfeed/feedtest/clock_test.go |
Tests virtual timer behavior. |
eventfeed/feedtest/minter.go |
Provides a scripted ticket minter. |
eventfeed/feedtest/minter_test.go |
Tests minter scripting and cancellation. |
eventfeed/feedtest/polls.go |
Provides a scripted poll source. |
eventfeed/feedtest/polls_test.go |
Tests poll scripting and cancellation. |
eventfeed/feedtest/store.go |
Provides a scripted checkpoint store. |
eventfeed/feedtest/transport.go |
Provides a scripted cable transport. |
eventfeed/feedtest/transport_test.go |
Tests fake connection behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
a2875be to
60a2870
Compare
Review of 4cff076 (B1 excluded, since uncommitted at the time). All four P1s reproduce; each fix is red-proven against the reported shape. P1 — Observer.Disconnected still leaked ticket text. Both arguments carry peer-controlled strings: a raw disconnect frame's reason, and a WebSocket close reason rendered through the error. Both were BOUNDED by §9's cap, which limits how much of a credential escapes rather than whether any does — the identical trap dialFailure documents three review rounds of. The cable server is exactly the party that knows the ticket: it was dialed with it. Both now go through closed vocabularies. observableDisconnectReason keeps the two reasons that change behavior and reports everything else as "other"; observableSocketError passes the connector's own sentinels and typed errors and degrades anything from a seam to a generic cause. CloseError.Error() renders only the code — an integer cannot carry a credential, and RFC 6455 codes are what an operator classifies on; Reason stays a readable FIELD. A canary planting a ticket in every peer-controlled teardown string found MORE than was reported: raw seam read errors leak too, which seam documentation cannot repair because the connector forwarded them verbatim. Four arms, all red before and green after. P1 — durableGate deadlocked reentrantly and blocked Close. It held the lock across CheckpointStore.Save while Close waited for it: a store whose Save calls Close self-deadlocks on the caller's own goroutine, and a merely stalled store blocked EVERY Close indefinitely — contradicting the one thing Close promises unconditionally. The two promises could not coexist, so the waiting one is dropped: the gate is claimed and released atomically, Close latches and returns, and a save that already claimed still completes. The guarantee is unchanged in substance — no save COMMENCES after Close returns — with commencing defined as claiming the gate, which takes no host code with it. The old test asserted Close WAITS and is replaced by one asserting it does not; the gate-holding variant deadlocks the new test at 40s. P1 — Close precedence, reopened by #763. Arming staleness before Connected also starts the pump before it, so a fatal frame can already be queued when a Connected callback calls Close, leaving two ready select cases. Reproduced: 25/50 rounds emitted a terminal element after Close returned. Fixed at the ONE exit (emitTerminal) rather than per-select — many selects, one exit, and a rule every future select must remember is what produced this. P1 — B2 discarded an earlier socket verdict. A deferred protocol-fatal followed by a positionless page took poll_failed, because disposal clears the deferral. The failed-poll branch already dispatches the deferral first, with a comment giving this exact reason; the new guard did not follow it. Now it does. Also fixed: TestNoCheckpointSaveCommencesAfterClose was vacuous (it closed before the run reached a page) and now closes from Observer.PageDelivered, the callback immediately preceding the save; the cancellation check after the checkpoint load covers every result rather than only the failure, since a found-empty result became terminal and a successful one let the run fire Connecting after Close; and deliver()'s stale "no delivery begins after Close returns" claim is corrected in place — it is a check-then-act, and the honest guarantee is the one Close states. Two of these touch foundations files that belong to #777 — CloseError in seams.go and its test. They stay here because the leak is only observable through the loop's observer path, which is this PR's, and the canary that proves it lives here. TestCloseError_Message is INVERTED, not adjusted: it required Error() to render the peer's reason, so it pinned the wrong contract. Verified: build, vet, -race, 22/22 fixtures, go-lint 0 issues, gosec 0 issues.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 41 out of 42 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
go/pkg/basecamp/eventfeed/websocket_transport.go:414
- A concurrent repeat
Closereturns as soon asclosedis set, while the first call may still be waiting for the graceful handshake and has not canceledlifetime. Pending reads/writes can therefore remain blocked after thatClosehas returned, violating theCableConn.Closecontract. Publish a shared completion channel/result so every concurrent caller waits for the first teardown to finish.
go/pkg/basecamp/eventfeed/filestore.go:398 - This rename does not preserve the documented support for a symlink to a regular store file: atomic rename replaces the symlink itself, leaving its target unchanged. A later consumer opening the target sees the stale checkpoint, while this spelling sees a new unrelated file. Either reject symlink paths consistently or resolve and lock/write the target identity without breaking atomic replacement.
go/pkg/basecamp/eventfeed/websocket_transport.go:310 - The method documents cancellation before local-close precedence, but this early return reverses it when both happen before entry. That can turn a canceled operation into a socket failure;
WriteFramealready checksctx.Err()first. Apply the same ordering here.
Suppressed comments, round 2 — three findings, two verdicts and a stopCopilot's review on 1.
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 41 out of 42 changed files in this pull request and generated 3 comments.
Suppressed comments (1)
go/pkg/basecamp/eventfeed/websocket_transport.go:423
- Concurrent callers do not observe completion of the same close. The first caller sets
closedbefore starting/waiting for the handshake, so a second caller can returnnilimmediately while the connection lifetime is still active and pending I/O remains blocked for up to the close budget. SinceCloseis documented as safe from any goroutine and as unblocking reads/writes, make repeated callers wait on a shared close-completion signal (and return the completed result) instead of treating an in-progress close as complete.
Stopping here: this is the fourth round on one question, not two more patchesRound 3 on The pattern
That round ended by replacing the redactor with a closed vocabulary keyed on error types — and generalised it to exactly one call site. Two other observer-facing renderings were left on the older model, "compose once and bound by §9's
Round 3's two comments are the observation that truncation is not redaction, applied to those two sites. That observation is correct, and it is the same observation the earlier three rounds made. Four rounds, one question, two models live in one package. What I think the real question isIs Until that is answered, any patch here is the fourth selector on an instrument nobody has sized — and the two obvious local fixes are both wrong in an instructive way:
So this is a SPEC §23/§9 decision with a conformance-schema and six-SDK blast radius, not a Go-file fix, and it should not be made inside a review round on the foundations PR. Flagging it for a human call; the threads carry the same reasoning and are resolved so the PR is not held open on a decision that is not mine. On merit, for the recordThe findings are true but the actor is narrow: the entity that can trigger either is the cable server the mint pointed us at, which already holds the ticket — it received it in the handshake URL. Nothing is disclosed to a party that did not have it; what is at stake is our own short-lived credential landing in the operator's log aggregator. Real, worth fixing, and not urgent enough to justify guessing at the spec. Also in round 3
|
|
Tracked as #788, so the analysis above survives this PR's squash-merge rather than living only in a comment thread. The issue carries the question as posed here — whether Not holding this PR on it. The threads are resolved because the decision isn't this PR's to make. |
A stacked-PR failure mode worth writing down: a moving base can silently disable Copilot reviewRecording this on the base PR because the diagnosis is not discoverable from the symptom, and the next person to hit it will be looking at #777's history rather than at the child PR. Symptom. Copilot posts, in place of a review:
What actually happened. #705 is stacked on this branch. When
The child PR crossed a reviewer's size limit without a single line of its own changing. Rebasing Why it is worth a note rather than a shrug. The failure is silent in the direction that matters: Diagnosis, for next time. If a stacked PR's reviewer goes quiet or refuses on size, compare what GitHub thinks the diff is against what the branch actually carries: A large disagreement means the base moved. Both of this branch's moves have now been absorbed downstream: |
Review of 4cff076 (B1 excluded, since uncommitted at the time). All four P1s reproduce; each fix is red-proven against the reported shape. P1 — Observer.Disconnected still leaked ticket text. Both arguments carry peer-controlled strings: a raw disconnect frame's reason, and a WebSocket close reason rendered through the error. Both were BOUNDED by §9's cap, which limits how much of a credential escapes rather than whether any does — the identical trap dialFailure documents three review rounds of. The cable server is exactly the party that knows the ticket: it was dialed with it. Both now go through closed vocabularies. observableDisconnectReason keeps the two reasons that change behavior and reports everything else as "other"; observableSocketError passes the connector's own sentinels and typed errors and degrades anything from a seam to a generic cause. CloseError.Error() renders only the code — an integer cannot carry a credential, and RFC 6455 codes are what an operator classifies on; Reason stays a readable FIELD. A canary planting a ticket in every peer-controlled teardown string found MORE than was reported: raw seam read errors leak too, which seam documentation cannot repair because the connector forwarded them verbatim. Four arms, all red before and green after. P1 — durableGate deadlocked reentrantly and blocked Close. It held the lock across CheckpointStore.Save while Close waited for it: a store whose Save calls Close self-deadlocks on the caller's own goroutine, and a merely stalled store blocked EVERY Close indefinitely — contradicting the one thing Close promises unconditionally. The two promises could not coexist, so the waiting one is dropped: the gate is claimed and released atomically, Close latches and returns, and a save that already claimed still completes. The guarantee is unchanged in substance — no save COMMENCES after Close returns — with commencing defined as claiming the gate, which takes no host code with it. The old test asserted Close WAITS and is replaced by one asserting it does not; the gate-holding variant deadlocks the new test at 40s. P1 — Close precedence, reopened by #763. Arming staleness before Connected also starts the pump before it, so a fatal frame can already be queued when a Connected callback calls Close, leaving two ready select cases. Reproduced: 25/50 rounds emitted a terminal element after Close returned. Fixed at the ONE exit (emitTerminal) rather than per-select — many selects, one exit, and a rule every future select must remember is what produced this. P1 — B2 discarded an earlier socket verdict. A deferred protocol-fatal followed by a positionless page took poll_failed, because disposal clears the deferral. The failed-poll branch already dispatches the deferral first, with a comment giving this exact reason; the new guard did not follow it. Now it does. Also fixed: TestNoCheckpointSaveCommencesAfterClose was vacuous (it closed before the run reached a page) and now closes from Observer.PageDelivered, the callback immediately preceding the save; the cancellation check after the checkpoint load covers every result rather than only the failure, since a found-empty result became terminal and a successful one let the run fire Connecting after Close; and deliver()'s stale "no delivery begins after Close returns" claim is corrected in place — it is a check-then-act, and the honest guarantee is the one Close states. Two of these touch foundations files that belong to #777 — CloseError in seams.go and its test. They stay here because the leak is only observable through the loop's observer path, which is this PR's, and the canary that proves it lives here. TestCloseError_Message is INVERTED, not adjusted: it required Error() to render the peer's reason, so it pinned the wrong contract. Verified: build, vet, -race, 22/22 fixtures, go-lint 0 issues, gosec 0 issues.
b15e8d0 to
f241597
Compare
Review of 4cff076 (B1 excluded, since uncommitted at the time). All four P1s reproduce; each fix is red-proven against the reported shape. P1 — Observer.Disconnected still leaked ticket text. Both arguments carry peer-controlled strings: a raw disconnect frame's reason, and a WebSocket close reason rendered through the error. Both were BOUNDED by §9's cap, which limits how much of a credential escapes rather than whether any does — the identical trap dialFailure documents three review rounds of. The cable server is exactly the party that knows the ticket: it was dialed with it. Both now go through closed vocabularies. observableDisconnectReason keeps the two reasons that change behavior and reports everything else as "other"; observableSocketError passes the connector's own sentinels and typed errors and degrades anything from a seam to a generic cause. CloseError.Error() renders only the code — an integer cannot carry a credential, and RFC 6455 codes are what an operator classifies on; Reason stays a readable FIELD. A canary planting a ticket in every peer-controlled teardown string found MORE than was reported: raw seam read errors leak too, which seam documentation cannot repair because the connector forwarded them verbatim. Four arms, all red before and green after. P1 — durableGate deadlocked reentrantly and blocked Close. It held the lock across CheckpointStore.Save while Close waited for it: a store whose Save calls Close self-deadlocks on the caller's own goroutine, and a merely stalled store blocked EVERY Close indefinitely — contradicting the one thing Close promises unconditionally. The two promises could not coexist, so the waiting one is dropped: the gate is claimed and released atomically, Close latches and returns, and a save that already claimed still completes. The guarantee is unchanged in substance — no save COMMENCES after Close returns — with commencing defined as claiming the gate, which takes no host code with it. The old test asserted Close WAITS and is replaced by one asserting it does not; the gate-holding variant deadlocks the new test at 40s. P1 — Close precedence, reopened by #763. Arming staleness before Connected also starts the pump before it, so a fatal frame can already be queued when a Connected callback calls Close, leaving two ready select cases. Reproduced: 25/50 rounds emitted a terminal element after Close returned. Fixed at the ONE exit (emitTerminal) rather than per-select — many selects, one exit, and a rule every future select must remember is what produced this. P1 — B2 discarded an earlier socket verdict. A deferred protocol-fatal followed by a positionless page took poll_failed, because disposal clears the deferral. The failed-poll branch already dispatches the deferral first, with a comment giving this exact reason; the new guard did not follow it. Now it does. Also fixed: TestNoCheckpointSaveCommencesAfterClose was vacuous (it closed before the run reached a page) and now closes from Observer.PageDelivered, the callback immediately preceding the save; the cancellation check after the checkpoint load covers every result rather than only the failure, since a found-empty result became terminal and a successful one let the run fire Connecting after Close; and deliver()'s stale "no delivery begins after Close returns" claim is corrected in place — it is a check-then-act, and the honest guarantee is the one Close states. Two of these touch foundations files that belong to #777 — CloseError in seams.go and its test. They stay here because the leak is only observable through the loop's observer path, which is this PR's, and the canary that proves it lives here. TestCloseError_Message is INVERTED, not adjusted: it required Error() to render the peer's reason, so it pinned the wrong contract. Verified: build, vet, -race, 22/22 fixtures, go-lint 0 issues, gosec 0 issues.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.
Suppressed comments (1)
go/pkg/basecamp/eventfeed/seams.go:339
DialError.Error()bypasses the 500-byte cap that §23 says still applies to other error renderings.checkCableURLplaces the server-supplied scheme or explicit port inReason, andnet/urlaccepts arbitrarily long valid schemes, so a malformed mint response can produce an arbitrarily large observer/log message. Apply the package truncation helper to the composed result.
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Copilot: FlushFileBuffers rejects directory handles, so the post-rename directory sync turned every successful Windows replacement into a reported failure. The sync is skipped there with the reasoning on the branch (NTFS journals rename metadata; the file-content sync still runs everywhere). And mustParseURL lost its last caller in the proxy removal -- the Lint job caught it; removed with its import.
|
@codex review |
Copilot: the round-16 classifier folded case, so a wrong-case selection (a protocol this dial never offered, refused by the library before any conn exists) read as matching the offer and fell to transient -- the re-mint-forever shape the classifier exists to stop. Exact comparison, matching the accepted-connection check; the existing wrong-case pin drives this branch since the library refuses before returning a conn.
|
@codex review |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 43 out of 44 changed files in this pull request and generated no new comments.
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
go/pkg/basecamp/eventfeed/cable.go:378
json.Decoder.Tokendecodes numbers asfloat64unlessUseNumberis enabled. Therefore a valid forward-compatible frame such as{"type":"future","value":1e1000}passes theRawMessageunmarshal but fails this duplicate-key walk and is treated as an invalid frame, contrary to the contract that unknown parseable frame types are ignored. EnableUseNumberbefore tokenizing so the duplicate check accepts the full JSON number grammar.
dec := json.NewDecoder(bytes.NewReader(data))
go/pkg/basecamp/eventfeed/filestore.go:518
- Resolving the final symlink for writes while retaining a lock keyed by the configured spelling means a store opened through the link and one opened through its target mutate the same file under different mutexes. Concurrent read-modify-write saves can therefore silently drop one lineage, even though the symlink tests describe link- and target-addressed consumers as one store. The lock identity and write target need one consistent identity, or the concurrency guarantee must explicitly exclude this supported alias.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02a9b076c8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Three Codex findings, one file.
The duplicate-key walk counted string TOKENS, and a null position
contributes a key token but no value token — so one duplicated key (surplus
two) plus two null-valued entries (deficit two) balanced the
strTokens == 2*len(entries) equality and the duplicated lineage loaded
last-wins. Red first with exactly that file: six tokens, three entries,
Load returned ("pos-2", true, nil). Detection now counts MEMBERS with the
codec's topLevelMemberCount — value types cannot cancel anything — and a
null position itself stays priced by the empty-position rule at lookup.
The lock key lowercased instead of folding: ſ (U+017F) case-folds together
with S and s on APFS/NTFS — one physical file — while ToLower leaves ſ
alone, so two spellings took two mutexes and raced the read-modify-write
the registry exists to serialize. Red first: ſtore.json kept its own key.
Each rune now maps to the minimum of its unicode.SimpleFold orbit, covering
every one-rune fold; the honest edges — full-fold multi-rune expansions and
normalization — are named in the doc as deliberately unchased, with
byte-identity the documented guarantee and unseen aliases degrading to the
documented cross-process last-writer-wins.
And a failed filepath.Abs no longer falls back to the relative spelling —
the identity-split class in its purest form: a path that names a DIFFERENT
file after every later chdir while keeping the old spelling's lock. The
constructor keeps its signature; the store records the resolution error and
every Load and Save reports it before touching the filesystem, with a
private mutex since it serializes with nothing. The pin drives it by
removing the working directory; on this macOS Getwd still resolves a
removed cwd, so the test self-skips here and bites where the platform
allows — stated plainly rather than simulated around.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be970aafe4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…n it Codex asked for the dial path's closed-vocabulary flattening on ReadFrame's raw fallthrough, on the claim that a TCP read failure's net.OpError renders a server-selected address that can carry the ticket. The claim fails on structure, and the asymmetry with dialFailure is exactly the line this review has drawn all along. A dial error can wrap a *url.Error rendering the full ticket-bearing URL — unbounded server-chosen text, so the dial path flattens. A post-handshake read error cannot render any dialed-URL component: wsConn retains no URL at all (the leak is inexpressible even by mutation), an OpError's address is the RESOLVED IP plus the CONNECTED port — a number in 1-65535, which an opaque ticket cannot be, the dial-status decline's reasoning on an even harder boundary since the port also accepted a TCP connect — and every peer-chosen text channel in a read error is already mapped: close reasons through the withholding CloseError, the read limit through ErrFrameOversize. Flattening would spend the one genuinely diagnostic cause (reset vs timeout vs EOF) to remove text that cannot carry a credential, and the run loop's observer vocabulary reduces unrecognized read errors to its generic sentinel before any logging surface regardless. The fallthrough now says so where the next reviewer will look, and the channel is pinned the way the write path is: a tripwire that dials through a NAME (so resolution is exercised), kills the peer's TCP abruptly, and walks the read error's chain for the ticket, the query, and the dialed hostname. Today it logs "failed to read frame header: EOF" — library prose, nothing dialed; a future change that starts retaining or rendering the URL goes red here.
|
@codex review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Coordination note now that #802 has merged SPEC §9 and #837 carries the SDK conformance sweep: the boundary question that spent rounds 3–4 here (#788) is decided, and it lands lightly on this stack. Under merged §9 the rule is credential-scoped — the closed set of secrets the SDK holds or requested, not a general theory of peer-derived text. For the connector that means:
So no further reworking of those two renderings is owed by this PR, and future review rounds have a spec section to point at instead of re-litigating the boundary. #837 touches none of the §23 code. |
First half of the SPEC.md §23 event feed connector, split out of #705 so the
state machine can be reviewed on its own. #705 keeps the run loop, catch-up,
recovery and the tier-2 driver, and now stacks on this.
Eight bot rounds on #705 did not converge (12→3→5→2→2→1→3 threads, with late
findings in files no earlier round had touched), and a review pass found a P1
credential defect that all eight missed because it composes two files across a
package boundary. Splitting is the response to that shape.
Size: ~2.7k lines of production code, ~7k with tests.
What is here
Everything the run loop is built from and nothing that runs — each piece
testable without starting a feed:
seams.goTicketMinter/PollSource/CableTransport/CableConnevent.goEvent,Cursor,Page,Signal,Disposition,Observererrors.goTerminalErrorand its reason codesfilters.go,digest.gocheckpoint.goCheckpointKey/FlatKey/CanonicalOrigin, the store seamcontinuation.gofilestore.goFileCheckpointStorededupe.go,backoff.go,clock.go,cable.gotransport.go,websocket_transport.gofeedtest/There is no consumer entry point yet:
Newand the loop are on #705.The four fixes
1. The cable dial takes no credential it was not given (P1)
The cable origin is chosen by the server — the mint returns a url and the
connector dials it verbatim, cross-host by design — and the short-lived ticket
in its query is the only credential that origin is entitled to. Two paths handed
it more.
WebSocketTransport.HTTPClient, orhttp.DefaultClientwhen nil. An*http.Clientcarries three credentials invisible at the call site: aRoundTripper may inject
Authorization, a Jar attaches cookies, aTLSClientConfigmay present a client certificate. The DefaultClient fallbackis the same hazard with no call site at all. Deleted rather than validated —
a RoundTripper is opaque, so no runtime inspection could accept one client and
refuse another. Handshakes now run on a package-owned client. (The field had no
callers anywhere, so nothing regressed with it.)
URL userinfo.
net/http'ssend()turns it into a BasicAuthorizationheader, so a mint whose url carried userinfo made the connector authenticate
to a server-nominated origin with a credential the server chose. Refused before
any network I/O.
The proxy. A
wss://handshake reaches a proxy asCONNECT host:port, sothe ticket stays inside the tunnel; a
ws://handshake is forwarded in absoluteform, putting
/cable?ticket=…in the proxy's request line and access log inthe clear. Reachable, not theoretical: §9 admits
ws://for*.localhost, andnet/http's proxy rules exempt the literallocalhostand loopback IPs butnot
.localhostsubdomains. Cleartext dials no longer proxy; TLS dialsstill do.
Pre-fix transcript, against un-fixed code:
and from the proxy sentinel with the cleartext exemption removed:
TestCableHTTPClient_IsWiredShutexists because the first mutant writtenagainst the proxy fix survived:
Proxy: proxyFromEnvironmentcaptures thevar's value at init, so the behavior tests' sentinel never reached it — meaning
both would pass a regression to
Proxy: http.ProxyFromEnvironment. The wiringassertion holds the shape they cannot observe.
2. A suppressed duplicate is not a delivery
§23 defines the LRU as "actually-delivered event ids", recorded by every
delivery.
Seenrefreshed recency on a hit, which is the case where theevent is suppressed and no delivery happens. §23 says to expect poll-vs-push
duplication continuously, so a hot id was pinned at the front and evicted ids
delivered once and never seen again — which become eligible for exactly the
re-delivery the LRU prevents.
TestDedupe_HitRefreshesRecencyis inverted to..._HitDoesNotRefreshRecency,and I am calling that out rather than letting it look like a test edited to
accept a fix. It asserted the negation of the contract; nothing short of
inverting it is honest.
3. The checkpoint store reads a bounded regular file, under one lock
#761: the lock registry was keyed on the exact path spelling. On APFS or
NTFS
feed.jsonandFeed.jsonare one file, so two stores took two mutexes —the lost update the registry exists to prevent, reached by two call sites
disagreeing about capitalization. The lock key is now case-folded; the path each
store reads and writes is not.
The read followed whatever the path named. Against the pre-fix read, all four
cases fail:
The FIFO and device cases are hangs, so every assertion runs under a bound. Not
defensive dressing: the first draft bounded only the FIFO, and
/dev/zerotookthe package's 45s timeout with it, naming nothing.
4. One observer-safe URL redactor
The primitive #705 applies to every URL-bearing observer surface. Reduction is
via
CanonicalOriginrather than truncation at?, which matters for the casea naive redactor misses: userinfo is a credential in the authority, so
https://attacker:hunter2@evil.example/steal?ticket=…survives query-strippingintact and does not survive this.
Verification
Pristine worktree, one pass, clean tree before and after:
go build/go vet/go test -race -count=1/-count=5— all passmake go-lint— 0 issuesgosec -severity high -exclude-dir=pkg/generatedon the CI-pinned v2.23.0(module hash verified, not a scratchpad binary) — 0 issues
make check— exit 0Every fix was red-proven before it was written, and every test mutation-checked
after. Three tests were rewritten because mutation showed them vacuous.
One preparatory commit
1646f2c1frelocated the run-coupled declarations out ofcheckpoint.goandcontinuation.goso the two halves fall on file boundaries. The moved functionbodies are byte-identical. It also added direct tests for
checkContinuationand
Filters.clone, both of which were only reachable through a full run.Development history, review threads and proof lineage for every file here are on
#705, preserved at tag
pre-split/705-head.Five more from review
Copilot's rounds on this branch found five further defects; all five are fixed, each
red-proven against the un-fixed code first.
6c6e14f16typeis not a broadcast. A*stringgives the same nil for an absent key and a JSONnull, so{"type":null}was liveness-only while{"type":null,"identifier":…,"message":…}was delivered as an event — one wire value in two classes depending on its siblings. Presence is now decoded separately. A present-but-null type takes the ignore branch, not the reject branch: it names no type to recognize, which is §23's unrecognized-type case. BC3's push lane sends{identifier, message}with notypekey at all, checked against the current head of bc3 #9659, so the narrowing drops nothing real.fae998b16Closebounds the read, not just itself.closeGraceBudgetstoppedClosefrom waiting out the close handshake, but the socket is what releases a parked read and coder/websocket does not tear it down until its own 5s+5s ends — so a pendingReadFramestayed blocked four seconds afterClosereturned. Worse, a background read was uncancellable: the library installs its cancellation hook only when the read context has aDonechannel, andReadFrame(context.Background())is how a run loop parks a pump. The connection now owns a lifetime context every read and write derives from, cancelled onceCloseis done waiting — after the budget, never before, so the close frame is still written.60a28700dConnector, no constructor and no run loop here, and the docs said "runs the whole protocol". Now: foundations only, what has landed, both pending pieces, and that everything below documents the architecture they implement. AGENTS.md's row carried the same overclaim.791143207Savecrossing the cap renamed into place a file nothing can read again — and sinceSavereads before it writes,Savetoo. No in-band recovery; the operator must delete the file, discarding every other lineage's cursor. Reaching the cap is accretion, not an adversary: there is no delete, so a filter change leaves the old lineage in the file forever. Refusing degrades to the documented failed-save outcome instead of an unrecoverable one.b15e8d031ReadFramedocuments the precedence and honored it on the way out but not on the way in, so a cancelled read over a closed connection reported a connection failure — the shutdown a run loop performs.WriteFramealready checked the context first; the two disagreed. The assertion went in the shared transport contract, where thefeedtestfake already passed it and the real transport did not.Two findings were declined on merit with the reasoning in a comment rather than left open, and a third — the symlinked store path versus atomic rename — is flagged for a human call: it is the third round on one file, and every candidate remedy trades away a different documented property of what a store file's identity is.
Summary by cubic
Lays the foundations for SPEC §23 Event Feed: seams, wire types, default
github.com/coder/websockettransport, a file-backed checkpoint store, and deterministic fakes. This also hardens transport and storage paths to avoid leaks and torn files.Bug Fixes
Loadreturns a usage-coded error when any key component is invalid before lookup.Load/Savereport the error and avoid filesystem I/O to prevent identity drift acrosschdir.Migration
CableTransport; the default transport does not use proxies.Written for commit ead96a5. Summary will update on new commits.