Real BloomLibrary login in e2e tests, and a bulk-upload test (Notion 211) - #8306
Draft
JohnThomson wants to merge 7 commits into
Draft
Real BloomLibrary login in e2e tests, and a bulk-upload test (Notion 211)#8306JohnThomson wants to merge 7 commits into
JohnThomson wants to merge 7 commits into
Conversation
Every Bloom of one build shares one user.config in %LOCALAPPDATA%\SIL\Bloom\<version>, so a Bloom that an e2e test launched started from whatever the developer's Bloom, or the previous run, saved last (UI language, page zoom, the Bloom Library login) and left its own changes behind for them. One run found every factory template named in Turkish; another had to put the zoom back when it was done. Bloom now takes --user-settings-folder <path>. BloomSettingsProvider, a thin subclass of libpalaso's CrossPlatformSettingsProvider, keeps user.config in that folder instead, for this process only; without the argument nothing changes. The startup arguments are parsed before anything reads Settings.Default, because a provider fixes its location when it is constructed, and the "bring in settings from a previous version" upgrade is skipped for a named folder, which holds exactly what its owner put there. common/instanceInfo reports the folder in use. The e2e launch fixture gives every Bloom an empty user-settings folder inside the run's temp folder, so it starts from default settings and its settings die with the run, and it fails the launch if the Bloom that came up reports a different folder (a stale Bloom.exe would otherwise quietly share settings). bloomApp.userSettingsDir and helpers/userSettings.ts let a test read what Bloom saved. The visual-regression suite passes the same argument, so a local run starts from defaults as a fresh CI runner does. user-settings-isolation.spec.ts checks the machinery end to end. The automation debt entry for the shared user.config is paid down and removed; the login entry now says its per-instance half is done. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…st Case ID 211) Automates the manual "Bulk Upload Quick Test": a collection with no bookshelf is refused, a first upload sends every book, an unchanged re-upload skips them all, and changing one book updates only it. It is the first e2e test that signs in to Bloom Library and uploads for real, which is possible now that each test's Bloom keeps its login in a settings folder of its own (the private-user-settings work this branch is stacked on). Bloom production code: - LibraryPublishApi.RefuseUploadWhileRunningE2eTests now refuses only bloomlibrary.org under --e2e; the sandbox, dev.bloomlibrary.org, is allowed. So no automated click can ever publish where the public would see it, whatever account is signed in, while a test can still upload to dev. - BloomLibraryPublishModel.BulkUpload passes the running Bloom's --user-settings-folder and --e2e to the upload child process, so the child reads the same login, and under --e2e runs the child with no console window (nobody would close it) and starts it without bringing a window to the front. Test infrastructure (src/BloomE2E): - helpers/bloomLibraryAccount.ts: sign the test account (e2e-tester@example.org, password from BLOOM_E2E_TESTER_EMAIL_BLORG_PASSWORD) in to dev for real, the way the website does (Firebase then parse-server), and post it to external/login. Skips locally when the password is unset; fails on CI. - helpers/bloomLibraryServer.ts: find and delete the account's books on the sandbox, so a run leaves nothing behind (and clears a crashed run's leftovers). - helpers/bulkUpload.ts: drive the "Upload this collection" split button and read the child process's BloomBulkUploadLog.txt tally. - helpers/collectionSettings.ts, fixtures/launchBloom.ts: a collection can carry a subscription code and a bookshelf; TEST_ENTERPRISE_SUBSCRIPTION_CODE and TEST_BOOKSHELVES name the built-in Test enterprise subscription. - nightly.yml passes the account password secret to the e2e step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
# Conflicts: # src/BloomE2E/fixtures/launchBloom.ts
JohnThomson
commented
Sep 4, 2026
JohnThomson
commented
Sep 4, 2026
JohnThomson
commented
Sep 4, 2026
Contributor
Author
|
[Claude Fable 5.1 · preflight] Consulted Devin on 2026-09-04 14:24 UTC up to commit 63072dd. Findings: 2 Bugs + 1 Investigate flag. "Production uploads bypass e2e safeguard" — assessed not an issue (UI path refuses production under --e2e; the spawned child always gets -d dev), replied and resolved. "Bulk-upload child lacks lifecycle ownership" — acceptable (waitForBulkUploadResult blocks until the upload finishes before teardown; matches the pre-existing fire-and-forget), replied and resolved. "Spreadsheet preference escapes settings isolation" — real and left open for the developer: ExportImportFileFolder is the one setting without the CrossPlatform provider, so --user-settings-folder does not isolate it. No current e2e test does spreadsheet export/import, so nothing leaks today. CI (pr-automation) green. |
It was the one setting of 43 without a provider attribute, so it used the default .NET settings provider and was written to the shared user.config regardless of --user-settings-folder. Give it Bloom.BloomSettingsProvider like the rest, so an e2e run's spreadsheet export/import folder cannot escape the run's private settings folder. (Devin, PR #8306.) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
JohnThomson
commented
Sep 4, 2026
Contributor
Author
|
[Claude Fable 5.1 · preflight] Consulted Devin on 2026-09-04 16:18 UTC up to commit bbc401d (re-review after the isolation fix). The fix is in: ExportImportFileFolder now uses the shared provider, so the settings isolation is complete. Devin's new finding "Saved spreadsheet folder disappears" is the disclosed one-time reset of that setting, which the developer accepted when choosing this fix — replied and resolved. Its other three findings this round are re-detections of already-resolved threads (production-bypass and child-lifecycle were assessed non-issues; the isolation finding is what this commit fixed). No open findings remain. |
Program.StartupUserSettingsFolder and BloomSettingsProvider.UserSettingsFolder held the same string. Drop the Program mirror: ParseStartupPortArguments now sets BloomSettingsProvider.UserSettingsFolder directly, and its consumers (the skip-upgrade check, the startup summary, the bulk-upload child process) read it there, next to the provider that uses it. Pure refactor, no behavior change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The refactor that moved the folder onto BloomSettingsProvider made the arg parser set it as it goes, so a valid --user-settings-folder followed by a bad argument left the folder set while the launch was rejected -- startup would open (and possibly self-heal) that user.config before reporting the error. Clear it on the parser's error path so a rejected launch touches nothing. (Devin, #8306.) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…g each book's xmatter
Folds the retired Notion test case 212 ("Bulk Upload Across Bookshelves Applies Xmatter"),
whose steps are now the last lines of test case 211, into the bulk-upload e2e test. After the
one-book update, the test moves the collection to the second Test bookshelf and, at the same
time, to a different front/back matter pack, then uploads again. All four books are updated, and
the HTML Bloom actually uploaded to S3 shows each of them on the new shelf with the new pack,
though nobody opened the books after the pack changed: bulk upload brings each book up to date
before it decides whether the book changed. The first upload's books are checked the same way.
Helpers, so no test spells out how to do any of this:
- bookHtml.ts: xmatterPackInBookHtml and readXmatterPackOfBook read the pack from the one
<Pack>-XMatter.css a book's head links.
- bloomLibraryServer.ts: the two record queries share one function; records carry their
bookshelves and baseUrl; fetchUploadedBookHtml and getXmatterPackOfBookOnServer read the
uploaded copy from the sandbox's S3 bucket.
- bulkUpload.ts: uploadCollection runs one whole upload round and returns its tally, which every
round of the test now uses instead of repeating the sequence.
AUTOMATION-DEBT.md: the "login cannot be done for real" entry is paid down by this PR and removed,
as the house rules say; what is left, that the Sign in button's trip through the system browser
cannot be driven, gets its own entry. The work-in-progress table names the PR.
Observed on the way: after the move, the sandbox record lists both bookshelves, the new one and
the old one. Bloom sends only the current shelf's tag (BookUpload.UploadBookAsync drops earlier
bookshelf tags), so the server is keeping the old one. The test asserts only what the card asks,
that the books are on the new shelf, and records the question in a comment.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Bloom's end-to-end suite could not test anything that needs a real BloomLibrary.org login — above all, uploading books. Signing in went out to an external browser, and the login lived in machine-wide settings shared with the developer's own Bloom, so a test could sign in or out only by disturbing the developer's session. The same machine-wide
user.config(one per build version) also meant a test inherited and overwrote unrelated settings — UI language, page zoom — from whatever Bloom ran last. So the "Bulk Upload Quick Test" (Notion test case 211) and every other real-upload case stayed manual.Fix
--user-settings-folderargument pointsSettings.Default(theuser.config) at a named folder for that process only (BloomSettingsProvider). The e2e launch fixture gives every Bloom a private folder inside the run's temp tree, so a run starts from defaults and its settings — the BloomLibrary login included — die with the run; it fails the launch if a staleBloom.exeignores the flag.common/instanceInforeports the folder. Without the flag nothing changes.--e2e; production stays blocked.RefuseUploadWhileRunningE2eTestsnow refuses only bloomlibrary.org, so an automated run can upload to dev.bloomlibrary.org for real while no automated click can ever publish where the public would see it. The bulk-upload child process inherits the same private settings folder and login, and under--e2eruns windowless.BLOOM_E2E_TESTER_EMAIL_BLORG_PASSWORD(a repository secret in CI, skipped locally when unset).user.config, and the real-login gap), and records the one piece left: the Sign in button's trip through the system browser.Not automated, and split to a manual portion of the card: confirming each uploaded book on the website against the screenshot on its front cover, which is a human visual check.
Observed while automating the move to a second bookshelf: after the re-upload, the sandbox record carries both bookshelf tags, the new one and the old one. Bloom itself sends only the current shelf's tag (
BookUpload.UploadBookAsyncdrops earlierbookshelf:tags), so it is the server that keeps the old one. The test asserts only that the books are on the new shelf; whether the old tag should go is a question for the library team.Devin review
This change is