Skip to content

Invalidate stale latest resolution when a newer explicit version is requested - #1398

Merged
ije merged 3 commits into
esm-dev:mainfrom
steve02081504:feat/invalidate-latest-on-explicit-version
Sep 9, 2026
Merged

Invalidate stale latest resolution when a newer explicit version is requested#1398
ije merged 3 commits into
esm-dev:mainfrom
steve02081504:feat/invalidate-latest-on-explicit-version

Conversation

@steve02081504

@steve02081504 steve02081504 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Problem

After publishing a new version, the default (bare-name) URL such as https://esm.sh/<pkg>/... keeps resolving to the previously cached latest version until the npm query cache TTL (npmQueryCacheTTL, default 600s) expires. Requesting the new version explicitly (<pkg>@newver/...) builds and caches it, but does not make the default URL follow — the two resolutions live in separate cache entries (npm:<pkg>@latest vs npm:<pkg>@<version>).

Change

When an explicit-version request has been built successfully, and that version is newer than the currently cached latest resolution, invalidate the npm:<pkg>@latest and 404:<pkg>@latest cache entries. The invalidation is triggered from the build-success path in the router (invalidateDistTagCacheIfNewer), and the next bare-name request re-resolves the dist-tag through the registry version route (a single small package.json, not the full metadata), so it still follows npm's official latest dist-tag — no guessing involved.

Two design choices worth noting:

  • Invalidation happens only on build success, not at package-resolution time. Doing it at resolution time would let repeated explicit-version requests continuously force npm re-queries (the invalidation runs even on cache hits), and could point the default URL at a version whose build actually fails. Since a built version is cached and not rebuilt, a successful build invalidates at most once per version — it cannot be replayed to keep forcing registry queries.
  • Requests for an equal or older version, or non-exact specifiers (including latest itself), never touch the cache. A v/=-prefixed version is normalized first.

Benefits

  • Instant follow for package authors: right after publishing, request the new version explicitly once and the default URL picks it up immediately instead of waiting out the TTL.
  • Allows a larger npmQueryCacheTTL: since authors can refresh on demand, deployments can raise npmQueryCacheTTL (e.g. to 1h) to reduce registry query pressure without delaying how soon new releases become the default.

Tests

Added TestInvalidateDistTagCacheIfNewer covering equal, older, newer, non-exact and v-prefixed version requests.

@steve02081504

steve02081504 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Alternative considered: bump the latest cache locally, without asking npm

While implementing this I also considered a cheaper alternative: when an explicit-version request resolves to a version newer than the currently cached latest, just overwrite the local npm:<pkg>@latest entry with the requested version — zero extra registry requests.

I did not follow that path, for a semantic reason:

latest is npm's dist-tag, not "the highest semver". resolveSemverVersion resolves it via metadata.DistTags["latest"], so esm.sh deliberately follows the registry's tag. Locally bumping the cache would assume "the explicitly-requested version == npm's latest tag", which can diverge in two cases:

  • a release is published with a formal version number but an isolated tag (e.g. npm publish --tag beta on a non-prerelease version), or
  • the latest tag is manually pinned to a lower version.

A prerelease-suffix check (semver.Version.Prerelease() != "") would only catch the 2.0.0-beta.1 case, not tag-isolated formal versions — there is no way to know the tag from the version number alone. So to stay strict, the PR invalidates the cached entry and lets the next bare-name request re-resolve the dist-tag through the registry version route (/latest, a single small package.json, then TTL-cached). In practice that is one ~4KB request after an author explicitly requests a new version.

This also pairs well with a larger npmQueryCacheTTL: authors refresh on demand, so the server can query the registry less often.

Second alternative considered: ask npm for its dist-tags instead of comparing versions

The invalidation above compares the requested version against the cached latest resolution. We also looked at querying npm's dist-tags endpoint (GET /-/package/<pkg>/dist-tags, a small JSON object like {"latest":"2.0.0","beta":"2.1.0-beta.1"}) after a build and invalidating only when the requested version is exactly what some tag points to. That would in principle also cover the isolated-tag and pin-back cases above. Pin-backs are not hypothetical — maintainers move latest back to a lower version most often to roll back a poisoned or badly broken release (npm dist-tag add <pkg>@<safe> latest), sometimes as an LTS/stability policy.

We still did not go with it. The re-resolution already is the tag lookup: the bare-name URL goes straight through the registry's dist-tag route, so it cannot disagree with npm — which is the entire point. Querying the tags separately would cost the same single small request (it only happens once per freshly built version, on demand) yet add an endpoint, a matching branch and a failure path, to save at most a redundant re-resolution that is harmless and rare.

One honest caveat: because the invalidation fires only when the requested version is newer than the cached latest, a rollback that pins latest to a lower version is not followed by requesting that lower version itself — the cache then heals at the next TTL expiry. A fix released on top of the poisoned version (a strictly newer version) does trigger it; only a hard rollback to a lower version line relies on the TTL.

@steve02081504
steve02081504 marked this pull request as draft September 2, 2026 02:16
@steve02081504
steve02081504 marked this pull request as ready for review September 2, 2026 02:18

@ije ije left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ije
ije merged commit 1eb40d0 into esm-dev:main Sep 9, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants