Skip to content

feat(analytics): visitor sources — referrer + UTM, aggregate-only - #210

Open
thunpisit wants to merge 2 commits into
mainfrom
feat/visitor-sources-v2
Open

feat(analytics): visitor sources — referrer + UTM, aggregate-only#210
thunpisit wants to merge 2 commits into
mainfrom
feat/visitor-sources-v2

Conversation

@thunpisit

Copy link
Copy Markdown
Contributor

Adds a visitor sources view to the admin: where traffic comes from, over a 30-day window.

Storage shape is the privacy guarantee

visitor_sources (migration 0034) is keyed on the whole dimension tuple — (date, channel, source, medium, campaign, path) — with count as the only payload. Every landing UPSERTs into an existing bucket; there is no append path.

That is what makes journey reconstruction impossible rather than merely discouraged:

  • no id, no session column — nothing links two landings together
  • no timestamp finer than the UTC date — so rows cannot be sorted into a sequence
  • no IP, user agent, fingerprint or country
  • no full referring URL — a referrer arrives as https://www.google.co.th/search?q=<what they typed> and is reduced to the token google before anything is stored

A row says "N landings matched this description that day". It cannot say who, in what order, or whether N was one person N times or N people once. No session id was needed, and none was added.

Consent

Gated identically to trackView: no consent.analytics means no row and no D1 round-trip at all. A test counts prepare() calls on the D1 shim and asserts zero, so "write it then delete it later" cannot regress in.

Bounded cardinality

source, medium and campaign are partly attacker-supplied via utm_*. Each is lower-cased, held to ^[a-z0-9._+-]+$ and 64 chars, and bucketed to other otherwise — so a ?utm_source=<random> spray collapses into a single row (pinned by a test). Unknown referrer hostnames go through the same funnel.

Spoofing fix: the multi-TLD engine matcher used a substring test, so google.com.evil.example classified as organic Google traffic — anyone with a domain could forge a Referer and write their own source name into the reports. Matching now requires the engine label to own the registrable domain.

UI

A new page at /admin/analytics (editor+), not the dashboard:

  • /admin/reports is shop-plugin-gated, so general site analytics cannot live there — it has to survive a site that sells nothing
  • a concurrent branch is redesigning /admin/dashboard; this avoids the collision (the dashboard is untouched)
  • it is a five-table drill-down that wants wide, while the dashboard is a scannable card summary

Channel breakdown, top sources, top campaigns, top landing pages, and a 30-day series, via PageShell(wide) / PageHeader / DataTable. Internal navigation is excluded from the source and landing-page tables — moving between our own pages is not an acquisition — but stays visible as its own channel row.

Message keys added to both en and th, additions only.

Docs

docs/ANALYTICS.md — what is captured and, at length, what is deliberately not, plus the cardinality strategy.

Verification

Gate: lint 0 errors · check 0 errors · 1126 tests passing (1114 pre-existing + 12 new, 79 files) · build clean · contract-guard OK (additions only) · css-inventory OK (+1 class, no re-baseline).

Driven end-to-end against wrangler dev --local with real D1 and varied referrers/UTMs:

Property Result
No-consent request no row written
google.co.th/search?q=very+private+terms stored as source google — search terms absent
google.com.evil.example referral, not organic Google
8 × 80-char random utm_source collapsed to one other row, count 8
channels / campaigns / internal classified correctly

Not merged, not deployed.

🤖 Generated with Claude Code

thunpisit and others added 2 commits September 5, 2026 14:14
Covers the consent gate (no row AND no D1 round-trip), referrer
reduction to origin, channel classification, the aggregate shape, and
the utm spray cap.

Also fixes a spoofing bug the tests caught: the multi-TLD engine
matcher used a substring test, so google.com.evil.example classified as
organic Google traffic. Matching now requires the engine label to own
the registrable domain.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds a "where did they come from" view to the admin, built on the same
counter discipline as page_views rather than an event log.

Storage (`visitor_sources`, migration 0034) is keyed on the whole
dimension tuple — (date, channel, source, medium, campaign, path) —
with a count as the only payload. Every landing UPSERTs into an
existing bucket; there is no append path. That shape is the privacy
guarantee, not a policy on top of one: with no id, no session column
and no timestamp finer than the UTC date, a row cannot say who, in
what order, or whether N landings were one visitor N times or N
visitors once. Referrers are reduced to an origin and classified to a
single token before storage, so a visitor's search terms never reach
the database.

Consent-gated identically to trackView: no `consent.analytics` means
no row and no D1 round-trip at all — asserted by a test that counts
prepare() calls on the D1 shim, so "write now, delete later" cannot
regress in.

Cardinality is bounded because utm_* is attacker-supplied: source,
medium and campaign are lower-cased, held to ^[a-z0-9._+-]+$ and 64
chars, and bucketed to "other" otherwise. A spray of distinct random
values collapses to one row (pinned by a test). Unknown referrer hosts
go through the same funnel.

Also fixes a spoofing bug the salvaged tests caught: the multi-TLD
engine matcher used a substring test, so google.com.evil.example
classified as organic Google traffic. Matching now requires the engine
label to own the registrable domain.

UI is a NEW page at /admin/analytics (editor+), deliberately not the
dashboard — /admin/reports is shop-plugin-gated so general site
analytics cannot live there, and a concurrent branch is redesigning
the dashboard. Channel breakdown, top sources, top campaigns, top
landing pages and a 30-day series, via PageShell(wide)/PageHeader/
DataTable. Internal navigation is excluded from the source and
landing-page tables — moving between our own pages is not an
acquisition — but stays visible as its own channel.

Message keys added to both en and th (additions only). Docs in
docs/ANALYTICS.md cover what is captured and, at length, what is
deliberately not.

Verified against wrangler dev + real local D1: consent gate, channel
classification, the spoof case, and an 80-char utm_source spray
collapsing to a single "other" row.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant