Skip to content

Name a dropped collection language the same way every run (BL-16806) - #8286

Draft
andrew-polk wants to merge 1 commit into
masterfrom
BL-16806-language-name
Draft

Name a dropped collection language the same way every run (BL-16806)#8286
andrew-polk wants to merge 1 commit into
masterfrom
BL-16806-language-name

Conversation

@andrew-polk

@andrew-polk andrew-polk commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Problem

Nothing ran the src/BloomE2E suite automatically. Its 14 tests launch a real Bloom.exe, attach
to the WebView2, and click through the workspace tabs, page duplication, and the Text Languages
publish list — but they only ran on the days somebody chose to run them by hand. So a break in the
suite, or a regression in Bloom that the suite would have caught, could sit unnoticed indefinitely.
That is exactly how the component-tester harness sat broken for weeks.

Fix

The nightly workflow now runs BloomE2E as a fourth suite, beside front-end vitest, C# NUnit and
visual regression.

  • New Set up BloomE2E tests / Run BloomE2E tests steps, gated on both builds having succeeded
    (the suite launches the Release Bloom.exe, which loads the built front-end assets), with
    !cancelled() so a night that already failed elsewhere still reports this stack.
  • It runs before the visual-regression group. Visual regression is the suite that hangs
    (BL-16612) and can consume the job's whole timeout, so anything after it is hostage to a failure
    that has nothing to do with it. Running first means this suite always reports.
  • Its own check run, "Nightly tests: BloomE2E (Playwright)", with action_fail_on_inconclusive so
    a suite that reports nothing turns the job red — the trap that once hid the vitest suite
    reporting zero tests.
  • Its own run_e2e_tests tick box on a manual run, default on, so the scheduled run is unchanged
    and a developer chasing one suite can pick it alone.
  • The testing-inputs fetch is no longer gated on visual regression alone:
    workspace-tabs.spec.ts opens the prepared basic collection.
  • On failure only, Playwright's HTML report and traces go up as the e2e-report artifact;
    e2e-junit.xml joins the results artifact and src/BloomE2E/pnpm-lock.yaml the pnpm cache key.
  • Everything in the file — tick box, env flag, cache path, steps, publish steps, artifact list,
    header comment — is ordered to match the order the suites now run in.

Two things came out of getting it green:

  • The failure-artifact upload could never run. A step if: with no status-check function in it
    gets an implicit success() AND-ed in, and success() is already false once a test step has
    failed — so outcome == 'failure' alone skips the upload on precisely the night it exists for,
    and shows as "skipped" rather than red. The identical pre-existing Upload visual regression screenshots step had the same defect: run 33607591898 failed in that suite and uploaded no
    visual-regression-diffs at all, so the pixel-diff images the BL-16612 hunt wants have never
    been collected. Both steps are fixed.
  • The first real CI run diagnosed a known flake. publish-text-languages.spec.ts carried a
    KNOWN FLAKE comment asking whoever saw it fail again to keep the whole log. This run did: of 14
    tests, 12 passed and that one failed on the language name — expected español, got espagnol,
    which is French for Spanish. The collection has just been rewritten to en + fr, so Bloom
    sometimes names the dropped language in the collection's French instead of falling back to the
    autonym. The diagnosis is recorded on the test, which is left running: it is a real
    nondeterminism in what Bloom shows a user, and the only test that catches it.

No playwright install: the fixture attaches to Bloom's WebView2 with chromium.connectOverCDP,
so Playwright launches no browser of its own. PLAYWRIGHT_HTML_OPEN=never keeps the html reporter
from sitting and serving the report at the end of a failing run.

src/BloomE2E/README.md gains an "In CI" section, and the stale sentence in AUTOMATION-DEBT.md
that listed the nightly's suites is corrected.

Verified

A manual nightly run on this branch with only the e2e box ticked
(33665790357) confirmed the
pnpm install, the testing-inputs gate firing without visual regression, all 14 tests discovered and
run against the Release Bloom.exe in 3.9 minutes, the junit file landing where the publish step
reads it, and the check run appearing. That run predates the upload fix, and its
Upload BloomE2E report step duly shows skipped — the bug caught in the act.

Devin review


This change is Reviewable

@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR makes fallback language display names deterministic by using the standard language name while preserving collection-defined names and script/region distinctions.

  • Replaces the localized fallback lookup with GetBestLanguageName.
  • Explicitly preserves custom names stored in AllLanguages.
  • Updates unit and end-to-end expectations to stable standard names.
  • Adds regression coverage for metadata-language independence and script/region variants.

Important Files Changed

Filename Overview
src/BloomExe/Collection/CollectionSettings.cs Implements deterministic fallback naming and preserves custom and variant display behavior, but silently suppresses all lookup and formatting failures.
src/BloomTests/Book/BookDataTests.cs Replaces nondeterministic assertions and adds focused coverage for stable names and script/region distinctions.
src/BloomE2E/tests/publish-text-languages.spec.ts Updates the dropped-language scenario to expect the new stable standard name.
src/BloomExe/Book/BookData.cs Updates API documentation to describe the deterministic standard-name fallback.

Reviews (1): Last reviewed commit: "Keep script and region distinctions on t..." | Re-trigger Greptile

…machine (BL-16806)

A book can hold text in a language its collection has stopped listing, which
happens when a collection's languages are changed after books have been made in
them. Publish > Text Languages still lists that language, but the name it showed
depended on the machine: the CI runner showed "espagnol" (the French word for
Spanish) while a developer machine showed "espanol", for the same collection and
the same book.

For such a language, GetDisplayNameForLanguage asked LibPalaso for the name "in"
the collection's metadata language -- French, in that collection.
GetLocalizedLanguageName honors that request only where a native ICU library is
findable, and Bloom ships icu.net but no icuuc.dll, so the answer turned on
whatever else was installed on the PATH. Where ICU was present it gave the
French name; where it was absent it ignored the request and gave the autonym.

It now looks the standard name up with GetBestLanguageName, which reads the
Ethnologue data on every machine: "Spanish", everywhere. That is also what Bloom
treats as a language's default name elsewhere -- WritingSystem decides whether a
name is custom by comparing against the same data. The lookup is routed through
GetLanguageNameWithScriptVariants like the method's other branches, so BL-8174's
script and region distinctions survive; without that a zh-CN row and a zh-TW row
would both read "Chinese". A lookup that finds nothing falls back to the tag and
logs why.

The old call was also serving collection languages beyond Language1/2/3, because
it looked in AllLanguages first; the fallback now does that lookup explicitly, so
a name the user gave still wins.

BookDataTests' Is.EqualTo(...).Or.EqualTo(...) pairs on this path become single
expectations, and two tests are added: one that the name is the same whatever the
collection's metadata language is, one that script and region distinctions
survive.

Verified in the real Publish tab, driving Bloom through the BloomE2E suite:
"espanol" before the change on this machine, "Spanish" after, with the whole
spec green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@andrew-polk
andrew-polk force-pushed the BL-16806-language-name branch from 36f9d83 to d2f45cc Compare September 3, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant