Add a cache purge page and API with proof-of-work by steve02081504 · Pull Request #1405 · esm-dev/esm.sh

steve02081504 · 2026-09-10T00:49:58Z

Fixes #1222 — adds a cache purge page and API, the esm.sh equivalent of jsDelivr's purge tool.

What it does

GET /purge — a landing page where you enter an esm.sh URL or a bare specifier (pkg, pkg@version, @scope/pkg, gh/user/repo@ref).
POST /purge — drops every cached artifact of the resolved package/version: built modules & source maps, type declarations, build metadata (and its in-memory copy), the local npm store copy and the npm resolution/404 caches. The next request rebuilds it from scratch. The JSON response lists what was purged plus a URL to trigger the rebuild.
For non-exact specifiers (bare names / dist-tags / semver ranges) the cached latest resolution is invalidated first, so a stale latest cannot survive the purge — this complements the existing invalidateDistTagCacheIfNewer (Invalidate stale latest resolution when a newer explicit version is requested #1398).
Proof-of-work: every purge must solve a hashcash-style SHA-256 challenge from GET /purge/challenge (single-use, 2 min TTL, difficulty 4). Mass purge-and-rebuild attacks are not free; the page solves it automatically in the browser with Web Crypto, and the README shows a curl one-liner for scripts. A per-IP rate limit (5/min) is stacked on top.
Config: purgeCache (env PURGE_CACHE, default true) can disable the whole feature for self-hosted deployments.

The page is fully self-contained — no third-party code is loaded. The example chips only fill the input with specifiers like @steve02081504/async-eval and @steve02081504/virtual-console to try.

Tests

TestParsePurgeInput, TestPurgeRefreshDistTag, TestPowChallenge, TestPurgePackageCache (plain + scoped). Verified locally on Windows (go test ./server/, all green) and cross-compiled for Linux.

Note: the sibling PR #1406 makes server/ build/test on Windows — the reason go test can run here at all.

Add a /purge page (like jsDelivr's purge tool) plus a POST /purge API that drops every cached artifact of a package/version - built modules, type declarations, build metadata, the local npm store and the npm resolution caches - so the next request rebuilds it from scratch. For non-exact specifiers (bare names / dist-tags / semver ranges) the cached latest resolution is invalidated first, so a stale resolution cannot survive the purge. Every purge must solve a hashcash-style SHA-256 proof-of-work challenge (GET /purge/challenge, single-use, short TTL) so mass purge-and-rebuild attacks are not free, and is rate-limited per client IP. The page is fully self-contained and solves the challenge in the browser with Web Crypto; the README documents a curl flow for scripts.

- Drop the redundant dist-tag refresh: purgePackageCache already clears the \latest\ resolution entry, so the pre-parse refresh duplicated it - Drop ExpiresAt from the challenge response (no client reads it) and the per-challenge difficulty (a package-level constant) - crypto/rand.Read never fails since Go 1.24: drop the error handling - Garbage collect pending challenges only once the store runs full - Drop the empty id/nonce guard in powVerify (a map miss covers it) and use slices.DeleteFunc in the rate limiter - Initialize the purge response slices so empty results serialize as [] instead of null (the page then drops its defensive fallbacks) - Fix the purgeCache config: a JSON \alse\ was silently re-enabled when the PURGE_CACHE env var was unset; mirror the Compress mode - Purge page: inline single-use variables, share the monospace font stack via a CSS variable, clearer element names

\parseEsmPath\ already reports whether the request pinned an exact version; pass it through so a fixed-version purge leaves the unrelated \pkg@latest\ resolution cache alone (it would only cause an extra registry query on the next bare-name request). A purge of a bare name, dist-tag or semver range still drops the dist-tag entry so the target gets re-resolved.

ije · 2026-09-10T04:46:32Z

a, i think we need an github oauth for the purge api
b, needs to integrate with cloudflare cdn purge api

- Add an optional GitHub OAuth gate on POST /purge (githubClientId / githubClientSecret or GITHUB_CLIENT_ID / GITHUB_CLIENT_SECRET). The proof-of-work challenge is still required on top of the login, so the two form a double gate; the per-client rate limit is keyed by GitHub account once signed in. - Add /purge/login, /purge/callback, /purge/logout and /purge/auth.json, with a signed (HttpOnly, SameSite=Lax) session cookie; the page shows a Sign in with GitHub button and solves the challenge as before. - Add optional Cloudflare edge-cache purging (cloudflareZoneId / cloudflareApiToken or CLOUDFLARE_ZONE_ID / CLOUDFLARE_API_TOKEN): after a purge the public URLs derived from the removed storage keys are evicted via the purge_cache API in batches of 30. - Docs in the README; tests for the session, the OAuth flow and the Cloudflare batching.

steve02081504 · 2026-09-10T07:51:03Z

Implemented both in the latest commit (71117a2). Since a login alone doesn't stop throwaway accounts, I stacked the gates instead of swapping them: POST /purge still needs a solved proof-of-work challenge and, when configured, a signed-in GitHub user.

a) GitHub OAuth

Config: githubClientId / githubClientSecret (env GITHUB_CLIENT_ID / GITHUB_CLIENT_SECRET). Unset → behaves exactly as before (PoW only), so Add a cache purge page and API with proof-of-work #1405 stays usable for anyone self-hosting without an OAuth app.
Endpoints: GET /purge/login → GitHub authorize, GET /purge/callback (code → token → /user), GET /purge/logout, GET /purge/auth.json (the page uses it). The session is a signed HttpOnly/SameSite=Lax cookie (24 h), scope read:user only — no server-side session store.
Once signed in the rate limit is keyed by the GitHub account instead of the client IP.
The page shows a Sign in with GitHub button when the gate is on.

One design question worth your call: this authenticates who someone is but doesn't authorize what they may purge — any GitHub account can still purge anything (now attributable/bannable). Restricting to the package owner isn't generally possible: npm ownership can't be verified through GitHub OAuth, and for gh/user/repo@ref we'd need a repo-scoped token plus a permission check. Easy to add that check for the gh/ case if you want it.

b) Cloudflare

Config: cloudflareZoneId / cloudflareApiToken (env CLOUDFLARE_ZONE_ID / CLOUDFLARE_API_TOKEN). Unset → no-op.
After a purge, the public URLs derived from the removed storage keys are evicted via POST /client/v4/zones/{zone}/purge_cache, batched at 30 files per request (the standard-plan limit). Failures are logged but never fail the purge.
Limitation: non-Enterprise plans only purge exact URLs, and paths whose public form can't be reconstructed (build args are hashed into x-<sha1> path segments) are skipped — prefix/tag/hostname purging needs Enterprise. So a few query-permuted edge entries can survive until their TTL. If you'd rather just purge_everything per package, say the word.

Tests: TestPurgeSession, TestPurgeOAuthLoginRedirect, TestPurgeOAuthCallback{,RejectsBadState}, TestCdnPurgeURLs, TestCloudflarePurge (GitHub/Cloudflare are mocked with httptest). go test ./server/ is green locally (with the Windows fix from #1406; the environment blocks the network-dependent TestGhInstall here).

Happy to adjust any of these choices — any-account vs owner-only, exact-URL vs purge-everything, or anything else — or feel free to push to the branch directly.

ije · 2026-09-10T07:54:05Z

nice! can you add these env vars in the config as well?

steve02081504 · 2026-09-10T11:53:18Z

Done in a868865 — added the five settings to config.example.jsonc (purgeCache, githubClientId/githubClientSecret, cloudflareZoneId/cloudflareApiToken), each with a comment noting the matching env var, so the JSON config keys and the env vars are both discoverable.

steve02081504 · 2026-09-10T12:13:46Z

Addressed both points from the review in fc274ba (+ 1219c34).

Generic proof-of-work endpoint. The PoW machinery moved out of purge.go into server/pow.go, so it is no longer purge-specific:

GET /pow/challenge?scope=<scope> is the generic endpoint; scopes are registered in a powPolicies map with their own difficulty/TTL (currently just purge: 4 hex zeros / 2 min).
A challenge is now bound to its scope and stores its own difficulty, so powVerify(scope, id, nonce) uses the value the challenge was minted with, and a solution cannot be replayed against another endpoint.
POST /purge verifies with the purge scope; /purge/challenge is kept as a backward-compatible alias that mints a purge challenge. A future gated endpoint only needs to register a scope.

Shared page assets. The existing /embed/* route already serves server/embed/ with proper MIME types, so the static pages can share files:

server/embed/shared.css — reset, palette vars, dark color-scheme, base body. index.html and purge.html now <link> it instead of duplicating that block (README images already use /embed/).
server/embed/shared.mjs — solvePow + fetchPowChallenge; the purge page imports it instead of inlining the solver.

Tests: TestPowChallenge now also covers scope binding and unknown scopes, and a new TestPowChallengeRoute covers the generic endpoint, the alias, the unknown-scope 400, and the two shared assets. go test ./server/ passes locally apart from the pre-existing network/os-bound TestGhInstall and TestInstallCjsModuleLexerRetry.

- drop the cached resolution by package prefix instead of a single guessed version, so ranges, dist-tags, date specifiers and git refs are all re-resolved after a purge - parse the purge input with net/url instead of hand-rolled string surgery - reuse the shared TTL cache for the purge rate limit, so idle clients are reclaimed - drop the unused /purge/challenge alias and dedupe the GitHub OAuth requests

The * external-all variant is stored under a normalized .../ea/ segment that no longer nests under the plain package id for scoped/gh/pr packages, so its files and metadata survived a purge. Delete both namespaces and recover the un-normalized build path (the meta key) from the storage key.

A bare name, dist-tag, range, date or git ref only asks to re-check which version is current, so a purge now stops at invalidating the resolution cache and keeps the existing build; the next request rebuilds it only if the resolved version/commit actually moved. Purging the build outputs, types, npm store and CDN entries stays reserved for an explicit exact version.

ije

please remove the shared js and css, then i will merge! thanks for your great work!

react pulls a large dependency graph, so purging it triggers an expensive rebuild for what is only a demo. The remaining examples stay lightweight.

steve02081504 mentioned this pull request Sep 10, 2026

Make the server package build and test on Windows #1406

Open

steve02081504 force-pushed the feat/purge-cache-page branch from 9000e3d to a209d0a Compare September 10, 2026 00:58

steve02081504 added 3 commits September 10, 2026 09:02

Ensure files end with a newline

5ff74b5

Document the purge env vars in the config example

a868865

steve02081504 added 2 commits September 10, 2026 20:13

Generalize the proof-of-work endpoint and share page assets

fc274ba

Note how to run server unit tests on Windows

1219c34

steve02081504 added 2 commits September 10, 2026 20:31

steve02081504 mentioned this pull request Sep 10, 2026

/purge: key build metadata off the module path so a purge can remove it #1409

Open

ije reviewed Sep 10, 2026

View reviewed changes

Comment thread server/embed/shared.css Outdated

ije approved these changes Sep 10, 2026

View reviewed changes

steve02081504 added 3 commits September 10, 2026 23:22

Inline the shared page assets

12f9b7e

Drop react from the purge page examples

9a1f3df

react pulls a large dependency graph, so purging it triggers an expensive rebuild for what is only a demo. The remaining examples stay lightweight.

Simplify purge helpers and fix a flaky session test

f9e0be7

Navigation Menu

Uh oh!

Uh oh!

Add a cache purge page and API with proof-of-work - #1405

Add a cache purge page and API with proof-of-work#1405
steve02081504 wants to merge 14 commits into
esm-dev:mainfrom
steve02081504:feat/purge-cache-page

steve02081504 commented Sep 10, 2026 •

edited

Loading

Uh oh!

ije commented Sep 10, 2026

Uh oh!

steve02081504 commented Sep 10, 2026

Uh oh!

ije commented Sep 10, 2026

Uh oh!

steve02081504 commented Sep 10, 2026

Uh oh!

steve02081504 commented Sep 10, 2026

Uh oh!

Uh oh!

ije left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

steve02081504 commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

Tests

Uh oh!

ije commented Sep 10, 2026

Uh oh!

steve02081504 commented Sep 10, 2026

Uh oh!

ije commented Sep 10, 2026

Uh oh!

steve02081504 commented Sep 10, 2026

Uh oh!

steve02081504 commented Sep 10, 2026

Uh oh!

Uh oh!

ije left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

steve02081504 commented Sep 10, 2026 •

edited

Loading