Shadow location with a fixed stub in every WPT parity leg - #34
Merged
Conversation
The wpt-env location stub was installed only when the global was absent, so the Chromium baseline leg evaluated test names against the real page origin — the run-browser static server's per-run ephemeral port — while the round-trip guest used the stub. The three Create-invalid-urls names that interpolate location then keyed differently in every run and never matched across legs: the recorded loss set went stale on the next run, the same three tests surfaced as phantom losses, and their stub-named round-trip passes tripped the unmeasured-pass hard failure. (Issue #26 attributed the embedded port to the echo server; echod's URL only ever reaches test bodies, never names.) Declare the stub as a module-scoped const instead: the generated group modules prepend wpt-env.js at module scope, so the lexical binding shadows window.location in a browser page and every leg evaluates identical names by construction. Record the now-stable Chromium ratchet (empty — the phantom losses dissolve; Chromium's two lenient constructor validations remain visible as non-gating exceeded notes), and drop the blocked-gate notes: the Chromium legs stay out of CI only for want of a Chrome 137+ binary on the runner. Verified: just wpt::parity green and byte-identical Node leg records; just wpt::parity-chromium green twice in a row across different ephemeral ports. Fixes #26
This was referenced Aug 6, 2026
Adopt the webcrypto sibling's pattern: the parity browser legs always launch Playwright's own Chromium build, pinned by playwright-core's version in the parity lockfile, so losses-chromium.js measures one engine everywhere — local runs and CI alike — and a Chromium behavior shift arrives only with a deliberate playwright bump, never with a runner-image update. run-browser.mjs drops the system-Chrome discovery (findChrome stays with the conformance jco-browser leg, which prefers the runner's Chrome on purpose). The conformance job installs the pinned browser with a cache keyed on the parity lockfile (restore-keys let a bump download only what changed; --with-deps only on a cache miss), and conformance-checks gains the wpt::parity-chromium step. Verified: just wpt::parity-chromium green on the pinned build (Chrome for Testing 151 / chromium-1234) with the same empty loss set; just ci green end to end. Closes #35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #26 — with two corrections to that issue's rationale, detailed in
the review comment:
the per-run port in the unstable keys was the browser-leg page
server's origin (echod's URL only ever reaches test bodies), and the
"three Chromium losses" were a keying artifact — the round trip passes
the same tests under the stub's names, which also tripped the
independent unmeasured-pass hard failure.
Change
wpt-env.jsdeclares itslocationstub as an unconditionalmodule-scoped
constinstead of a conditional global assignment. Thegenerated group modules prepend wpt-env.js at module scope, so the
lexical binding shadows
window.locationin a browser page and everyleg — Node, Chromium page, componentized guest — evaluates identical
test names by construction. The stub's values are wptserve's canonical
host:port, so the names also match upstream WPT's.
With names unified, the Chromium ratchet records stably and empty:
losses-chromium.jsis committed with no entries. Chromium's twolenient constructor validations (
ws://foo bar.com/andwss://foo bar.com/do not throw) remain visible as non-gatingexceedednotes. The blocked-gate notes come off the justfile,run-browser.mjs, and README; the Chromium legs stay out of CI only for
want of a Chrome 137+ binary on the runner (unchanged, and a separate
decision).
Only behavioral delta: the Chromium baseline builds Create-invalid-urls'
negative-case URLs from the stub origin instead of the real page origin,
immaterial for must-throw tests where the scheme decides.
locationhasexactly two consumers in the tree (the stub and Create-invalid-urls).
Verification
just wpt::paritygreen; the two Node leg record files remainidentical on the affected group.
just wpt::update-losses-chromiumrecords 0 losses.just wpt::parity-chromiumgreen twice consecutively across differentephemeral ports (35393, 39895): 0 losses, 0 setup-failure renames, no
unmeasured passes.