Control over where automation and e2e runs (BL-16804)

.github/skills/add-e2e-test/SKILL.md

@@ -338,9 +338,19 @@ pnpm exec playwright test tests/workspace-tabs.spec.ts   # one file
 pnpm exec playwright test -g "switching workspace tabs"  # one test by title
 ```
 
-A run opens a real Bloom window; that is expected. It needs a built `Bloom.exe` under
-`output/{Debug,Release}/{x64,AnyCPU,}/` (build it yourself; see "Build Bloom whenever it
-helps") and the inputs at `output/testing-inputs`. Point
+A run launches a real Bloom and its window appears on the developer's desktop, unless
+`BLOOM_AUTOMATION_MONITOR` says otherwise. That one variable decides where every window a run
+opens goes, the splash screen included: `headless`, or `0`, puts them all off every monitor; a
+1-based monitor number puts them on that monitor; and any other value, unset included, leaves Bloom
+to place them as it always does. `headless` moves the window off-screen rather than minimizing it,
+because WebView2 stops painting a minimized window and every screenshot then comes back blank.
+`--debug` clears a `headless` setting, so a debug session has a window to step through. The monitor
+number counts left to right, so 1 is the leftmost monitor; it is **not** the number Windows Settings
+shows beside each display, and no API reproduces those. Bloom writes the whole mapping to its log at
+startup. See `src/BloomE2E/README.md` for the table and that log line.
+
+A run needs a built `Bloom.exe` under `output/{Debug,Release}/{x64,AnyCPU,}/` (build it yourself;
+see "Build Bloom whenever it helps") and the inputs at `output/testing-inputs`. Point
 `BLOOM_TESTING_INPUTS_DIR` at a bloom-testing-inputs checkout to use your own in-progress
 collections instead of the pinned ones.
 

.github/workflows/nightly.yml

@@ -273,6 +273,12 @@ jobs:
                   # The html reporter would otherwise try to serve the report at the end of a
                   # failing run, which on a runner means a step that never returns.
                   PLAYWRIGHT_HTML_OPEN: never
+                  # Put every window this run opens off every monitor. Bloom obeys this variable
+                  # only under --automation, which is what the fixture launches; see
+                  # src/BloomE2E/README.md. The runner has a desktop, so a visible window would
+                  # also work, but asking for it here says what the run needs instead of leaving
+                  # it to whatever the runner happens to do.
+                  BLOOM_AUTOMATION_MONITOR: headless
               run: pnpm test --reporter=list,junit,html
 
             # What explains an e2e failure: Playwright's HTML report, plus the trace and failure

src/BloomE2E/AUTOMATION-DEBT.md

@@ -287,3 +287,100 @@ the file said `en` again a moment later. The same test has to restore the zoom i
 that setting is shared too. Fix direction: under `--e2e`, point the settings provider at a
 per-instance folder (a sibling of the temp collection would do), so a test's Bloom starts from
 defaults and its changes die with it.
+
+## No way to run the suite at a chosen monitor resolution and scale factor
+
+Every run takes the resolution and the scale factor of whatever monitor it lands on, so a
+suite proves the layout only at the DPI of the machine that ran it. That is exactly where a
+class of Bloom bugs lives: a control that fits at 100% and overlaps at 150%, a dialog that
+opens off the edge on a short screen, a size computed in one coordinate space and used in
+another. A developer at 150% and a CI runner at 100% each pass while the other's bug goes
+unseen, and neither can reproduce what a user reports.
+
+Found 2026-09-03, twice in one change (BL-16804), which is what makes this worth scheduling:
+
+- The off-screen window asked for the primary monitor's working-area size, and Windows
+  interpreted that size at the scale factor of the nearest monitor. On a machine with a 150%
+  primary and a 100% monitor beside it, a window meant to be 3840x2100 came out 3840x2100 real
+  pixels, taller than any monitor on the machine. `format-gear-positioning.spec.ts` failed
+  because the page viewport was 1990 CSS pixels high, a size no user has. The same mismatch,
+  in its first guise, had eaten all but 27 pixels of a 1000-pixel off-screen cushion.
+- Both bugs passed every unit test, because a unit test compares numbers inside one process's
+  own coordinate space. Only a real window at a real scale factor shows them.
+
+Fix direction, cheapest first, none of it tried yet:
+
+- **An RDP session to the machine.** An `.rdp` file takes `desktopwidth`, `desktopheight` and
+  `desktopscalefactor` (100, 125, 150, 175, 200), so one connection per combination gives a
+  real desktop at a chosen scale with no driver to install. This looks like the least work and
+  the most likely to run in CI, but nobody has tried driving the suite inside one.
+- **A virtual display driver.** Windows has an indirect-display driver model (IddCx), and
+  several drivers built on it create a monitor with no hardware behind it, at a resolution the
+  driver is told to offer. Setting that monitor's *scale factor* is the harder half: Windows
+  exposes per-monitor scale only through display-config calls Microsoft does not document.
+  Worth an afternoon of investigation before committing to it.
+- **A virtual machine or Windows Sandbox** at a chosen resolution and scale. Heaviest, but it
+  is the only one that also isolates the shared `user.config` described above.
+
+Whatever the mechanism, the suite needs the same thing from it: a way to say "run these tests
+at 1920x1080 at 150%" and have the run either honour it or refuse, rather than silently using
+the desktop it found.
+
+One piece of this is a known limit in the code already, and it is what the fix direction above
+would settle. `AutomationWindowPlacement.GetBoundsOffEveryMonitor` puts an off-screen window
+directly below the primary monitor, because the nearest monitor is the one whose scale factor
+Windows applies, and on the layouts we have that keeps the primary nearest. It stops being true
+when a monitor sits *below* the primary in the same band of x: that lower monitor is then nearest,
+and if its scale factor differs the window comes out the wrong size, which is the same bug in a
+new layout. Fixing it properly means asking Windows for the nearest monitor's scale factor and
+scaling the requested size by the ratio, which needs the per-monitor DPI calls this entry is
+about. Nobody on the team has such a layout today, which is why it is written down rather than
+fixed. (Devin raised it on PR 8285, 2026-09-03.)
+
+## Every run takes the developer's window size, so small-screen bugs go unseen
+
+A run makes its window as big as the monitor it lands on, so the suite proves the layout only at
+the size of a developer's screen. Many Bloom users are on inexpensive machines with small screens,
+and that is where a class of bugs lives that nobody on the team meets: a control that overlaps
+another, a dialog that opens past an edge, a toolbar that quietly drops an item. This is the
+window-size half of the DPI entry above, and it is much cheaper to fix, because it needs no
+virtual monitor.
+
+The plan: give every automation run a window of **1024x586**, the working area of a 1024x768
+screen once a task bar of the usual height is taken off, wherever the window goes.
+`BLOOM_AUTOMATION_WINDOW_SIZE=1600x900` asks for a different size, for chasing a bug that only
+shows on a big screen. The floor is 400x300, which is `Shell.MinimumSize`; anything Bloom cannot
+use, a typo included, gives the default rather than a broken run. The size must be the same for
+all three values of `BLOOM_AUTOMATION_MONITOR`, so that variable decides only *where* a window
+goes: a suite whose size changed with its placement would let one test pass in one mode and fail
+in another, a trap that caught this code twice on BL-16804.
+
+The work is not the window size, which is about thirty lines in `AutomationWindowPlacement.cs` and
+`Shell.cs`. The work is the suite going red, which is the point of the change. One full run of the
+35 tests at 1024x586 on 2026-09-03 gave **16 passed, 6 failed, 13 did not run**, against 23
+passed and 2 failed at the size of a developer's monitor. Two of the six fail at either size, so
+they are not the window's doing: Test Case ID 349 (BL-16807) and Test Case ID 606, which times out
+after 60 seconds waiting for the publish-to-web steps. The small window is what added these four:
+
+- `copy-page.spec.ts:85` (Test Case ID 348), failed in 8 seconds.
+- `derivative-keeps-template-pages.spec.ts:106` (Test Case ID 72), failed after 48 seconds.
+- `format-gear-positioning.spec.ts:130` (Test Case ID 356), failed in 335 ms: the Format dialog no
+  longer opened close to its gear, while the test above it in the same file passed. So the small
+  window moved the dialog.
+- `publish-text-languages.spec.ts:416` (Test Case ID 169), failed after 37 seconds. Read this one
+  with care: it is BL-16806, which is machine-dependent, and it passed in the full-size run of the
+  same build. So the window may have caused it or may not.
+
+Because the suite is serial per file, those 6 failures also stop 13 more tests from running, so
+the small window costs 7 passes and hides 13 results until the fixes land.
+
+Each failure then needs triage into one of two piles, and the second pile is the reason to do any
+of this: either the test assumed a large window and has to be rewritten, or **Bloom itself
+misbehaves at 1024x586**, which is a real user-facing bug and wants its own card. Timeouts rather
+than quick failures are the common failure mode, so the suite is also much slower while the fixes
+are outstanding. Whoever picks this up has to decide what the nightly workflow does in the
+meantime: run small and stay red, or stay large until the tests are fixed.
+
+(Written and measured on 2026-09-03 during BL-16804, then deliberately taken back out: the
+developer chose to record the plan here rather than carry a red suite. The code is not in the
+history, so rebuilding it from this entry is part of the job.)

src/BloomE2E/README.md

@@ -135,7 +135,62 @@ pnpm exec playwright test -g "switching workspace tabs"       # one test by titl
 pnpm exec playwright test --debug    # step through it
 ```
 
-A run opens a real Bloom window. That is expected; do not click in it.
+A run opens a real Bloom window on your desktop. That is expected; do not click in it.
+
+### Where the Bloom window goes: `BLOOM_AUTOMATION_MONITOR`
+
+One environment variable decides where every window an e2e run opens goes, the main window and
+the splash screen alike. Set it in your shell, or per run:
+
+| Value | What happens |
+| --- | --- |
+| `headless`, or `0` | Every window opens far outside every monitor. You see nothing, so a run can go on while you work. |
+| a 1-based monitor number, counted left to right | Every window opens on that monitor, so a run stays off the one you are working on. |
+| unset, or anything else | Bloom places its windows as it always does, and you see the run. |
+
+```bash
+BLOOM_AUTOMATION_MONITOR=headless pnpm test    # see nothing
+BLOOM_AUTOMATION_MONITOR=0 pnpm test           # the same thing: no monitor at all
+BLOOM_AUTOMATION_MONITOR=2 pnpm test           # on the second monitor from the left
+pnpm test                                      # wherever Bloom normally opens
+```
+
+A value Bloom cannot use, a typo or a monitor you do not have, counts as "anything else": you get
+a visible window, which is exactly what tells you the variable did not take effect. A setting that
+hid the window on a typo would leave you nothing to notice.
+
+**The number counts left to right, which is not the number Windows Settings shows.** Monitor 1 is
+your leftmost monitor, 2 the next one to the right, and so on, matching the arrangement picture in
+Windows Settings but not the numbers printed on it. Windows does not document how the Settings app
+makes those numbers, and no API reproduces them: on one three-monitor machine Windows Settings said
+1 (primary, centre), 2 (right) and 3 (left), while left to right is 1 (left), 2 (primary, centre)
+and 3 (right). So read the arrangement, not the numbers on it. Bloom also writes the whole mapping
+to `%TEMP%\SIL\Bloom\Log.txt` on every automation start:
+
+```
+BLOOM_AUTOMATION_MONITOR='2': every window goes on the monitor at {X=0,Y=0,Width=2560,Height=1440}.
+The monitors this process sees, numbered left to right as this variable numbers them (which is NOT
+how Windows Settings numbers them): 1=(-1920,601) 1920x1200, 2=(0,0) 2560x1440 primary,
+3=(3840,432) 1920x1200.
+```
+
+`headless` moves the window off-screen rather than minimizing or hiding it, because WebView2 stops
+painting a minimized window, which would make every screenshot blank. Off-screen the window paints
+exactly as it would in front of you, so rendering and keyboard input behave the same.
+
+It goes **below** your primary monitor, not off to one side, and that matters on a machine whose
+monitors run at different scale factors. Windows gives a window the scale factor of the monitor
+nearest to it. A window out to the left would take the leftmost monitor's scale factor while
+carrying a size measured in the primary's, and on a 150% primary beside a 100% monitor that made a
+window 3840x2100 real pixels, taller than any monitor on the machine, with a page viewport no user
+could have. Directly below the primary, the primary stays the nearest monitor and the size is
+right. See `AutomationWindowPlacement.GetBoundsOffEveryMonitor`.
+
+`--debug` clears a `headless` setting for you: stepping through a test whose window you cannot see
+is pointless. A setting that names a monitor is left alone, because that window is visible anyway.
+
+The variable applies only to a run under `--automation`, which is every e2e run and nothing else.
+A Bloom you start yourself is unaffected, however the variable is set.
 
 The suite needs a built `Bloom.exe` under `output/{Debug,Release}/{x64,AnyCPU,}/` and the test
 inputs at `output/testing-inputs`, fetched by `node build/get-testing-inputs.mjs` at the commit

src/BloomE2E/fixtures/launchBloom.ts

@@ -158,6 +158,26 @@ function samePath(a: string, b: string): boolean {
 
 const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
 
+/**
+ * The environment the Bloom we launch runs in. One variable decides where its windows go,
+ * BLOOM_AUTOMATION_MONITOR, and Bloom reads it itself (see AutomationWindowPlacement.cs):
+ * "headless" puts every window off every monitor, a monitor number puts them on that monitor, and
+ * anything else, the variable being unset included, leaves Bloom to place its windows as it always
+ * does. So the child inherits this process's environment untouched, with one exception.
+ *
+ * The exception is Playwright's --debug (which sets PWDEBUG): stepping through a test whose window
+ * nobody can see is pointless, so a debug session clears a "headless" setting, or the "0" that says
+ * the same thing, and gets a window. A setting that names a monitor is left alone, because that
+ * window IS visible.
+ */
+function environmentForBloom(): NodeJS.ProcessEnv {
+    const asked = process.env.BLOOM_AUTOMATION_MONITOR?.trim().toLowerCase();
+    if (process.env.PWDEBUG && (asked === "headless" || asked === "0")) {
+        return { ...process.env, BLOOM_AUTOMATION_MONITOR: "" };
+    }
+    return process.env;
+}
+
 /** What common/instanceInfo tells us about a running Bloom. Only the fields we use. */
 interface IInstanceInfo {
     editableCollectionFolder?: string;
@@ -362,12 +382,12 @@ async function startBloomOn(
     };
 
     // --e2e: skip the DEBUG "attach debugger now" prompt and suppress modal error dialogs.
-    // --automation: let this instance run alongside a Bloom the developer already has open.
-    const bloomProcess: ChildProcess = execFile(exe, [
-        findCollectionFile(collectionDir),
-        "--e2e",
-        "--automation",
-    ]);
+    // --automation: let this instance run alongside a Bloom the developer already has open, and
+    // let BLOOM_AUTOMATION_MONITOR say where its windows go (see environmentForBloom).
+    const args = [findCollectionFile(collectionDir), "--e2e", "--automation"];
+    const bloomProcess: ChildProcess = execFile(exe, args, {
+        env: environmentForBloom(),
+    });
     let exitStatus: { code: number | null; signal: string | null } | undefined;
     bloomProcess.stdout?.on("data", (d) => recordOutput(String(d)));
     bloomProcess.stderr?.on("data", (d) => recordOutput(String(d)));