feat(analytics): visitor sources — referrer + UTM, aggregate-only - #210
Open
thunpisit wants to merge 2 commits into
Open
feat(analytics): visitor sources — referrer + UTM, aggregate-only#210thunpisit wants to merge 2 commits into
thunpisit wants to merge 2 commits into
Conversation
Covers the consent gate (no row AND no D1 round-trip), referrer reduction to origin, channel classification, the aggregate shape, and the utm spray cap. Also fixes a spoofing bug the tests caught: the multi-TLD engine matcher used a substring test, so google.com.evil.example classified as organic Google traffic. Matching now requires the engine label to own the registrable domain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a "where did they come from" view to the admin, built on the same counter discipline as page_views rather than an event log. Storage (`visitor_sources`, migration 0034) is keyed on the whole dimension tuple — (date, channel, source, medium, campaign, path) — with a count as the only payload. Every landing UPSERTs into an existing bucket; there is no append path. That shape is the privacy guarantee, not a policy on top of one: with no id, no session column and no timestamp finer than the UTC date, a row cannot say who, in what order, or whether N landings were one visitor N times or N visitors once. Referrers are reduced to an origin and classified to a single token before storage, so a visitor's search terms never reach the database. Consent-gated identically to trackView: no `consent.analytics` means no row and no D1 round-trip at all — asserted by a test that counts prepare() calls on the D1 shim, so "write now, delete later" cannot regress in. Cardinality is bounded because utm_* is attacker-supplied: source, medium and campaign are lower-cased, held to ^[a-z0-9._+-]+$ and 64 chars, and bucketed to "other" otherwise. A spray of distinct random values collapses to one row (pinned by a test). Unknown referrer hosts go through the same funnel. Also fixes a spoofing bug the salvaged tests caught: the multi-TLD engine matcher used a substring test, so google.com.evil.example classified as organic Google traffic. Matching now requires the engine label to own the registrable domain. UI is a NEW page at /admin/analytics (editor+), deliberately not the dashboard — /admin/reports is shop-plugin-gated so general site analytics cannot live there, and a concurrent branch is redesigning the dashboard. Channel breakdown, top sources, top campaigns, top landing pages and a 30-day series, via PageShell(wide)/PageHeader/ DataTable. Internal navigation is excluded from the source and landing-page tables — moving between our own pages is not an acquisition — but stays visible as its own channel. Message keys added to both en and th (additions only). Docs in docs/ANALYTICS.md cover what is captured and, at length, what is deliberately not. Verified against wrangler dev + real local D1: consent gate, channel classification, the spoof case, and an 80-char utm_source spray collapsing to a single "other" row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a visitor sources view to the admin: where traffic comes from, over a 30-day window.
Storage shape is the privacy guarantee
visitor_sources(migration0034) is keyed on the whole dimension tuple —(date, channel, source, medium, campaign, path)— withcountas the only payload. Every landing UPSERTs into an existing bucket; there is no append path.That is what makes journey reconstruction impossible rather than merely discouraged:
https://www.google.co.th/search?q=<what they typed>and is reduced to the tokengooglebefore anything is storedA row says "N landings matched this description that day". It cannot say who, in what order, or whether N was one person N times or N people once. No session id was needed, and none was added.
Consent
Gated identically to
trackView: noconsent.analyticsmeans no row and no D1 round-trip at all. A test countsprepare()calls on the D1 shim and asserts zero, so "write it then delete it later" cannot regress in.Bounded cardinality
source,mediumandcampaignare partly attacker-supplied viautm_*. Each is lower-cased, held to^[a-z0-9._+-]+$and 64 chars, and bucketed tootherotherwise — so a?utm_source=<random>spray collapses into a single row (pinned by a test). Unknown referrer hostnames go through the same funnel.Spoofing fix: the multi-TLD engine matcher used a substring test, so
google.com.evil.exampleclassified as organic Google traffic — anyone with a domain could forge aRefererand write their own source name into the reports. Matching now requires the engine label to own the registrable domain.UI
A new page at
/admin/analytics(editor+), not the dashboard:/admin/reportsis shop-plugin-gated, so general site analytics cannot live there — it has to survive a site that sells nothing/admin/dashboard; this avoids the collision (the dashboard is untouched)wide, while the dashboard is a scannable card summaryChannel breakdown, top sources, top campaigns, top landing pages, and a 30-day series, via
PageShell(wide)/PageHeader/DataTable. Internal navigation is excluded from the source and landing-page tables — moving between our own pages is not an acquisition — but stays visible as its own channel row.Message keys added to both
enandth, additions only.Docs
docs/ANALYTICS.md— what is captured and, at length, what is deliberately not, plus the cardinality strategy.Verification
Gate: lint 0 errors · check 0 errors · 1126 tests passing (1114 pre-existing + 12 new, 79 files) · build clean · contract-guard OK (additions only) · css-inventory OK (+1 class, no re-baseline).
Driven end-to-end against
wrangler dev --localwith real D1 and varied referrers/UTMs:google.co.th/search?q=very+private+termsgoogle— search terms absentgoogle.com.evil.examplereferral, not organic Googleutm_sourceotherrow, count 8Not merged, not deployed.
🤖 Generated with Claude Code