fix(seed): load reference data even when migrations seeded a term - #869
Merged
Conversation
The development-mode startup seed calls ensure_seed_prereqs, which loaded core/lexicon.json only when lexicon_term was completely empty. Since b2c3d4e5f6a7 (transducer data maturity) the migration inserts a lexicon term of its own, so on a fresh database that emptiness check now sees one row and skips the real lexicon entirely. What followed: init_parameter tripped parameter_default_unit_fkey because 'ft' and 'dimensionless' were never inserted, then seed_all called random.choice on an empty organization category and died with "IndexError: Cannot choose from an empty sequence". FastAPI aborted startup, so the frontend's Cypress job sat on a readiness probe for its full 720s timeout and failed with exit 124. Both initializers leave existing rows alone, so ensure_seed_prereqs now just runs them. init_parameter skips names already stored rather than letting every re-run trip the unique constraint, and a new assert_lexicon_ready names the empty categories instead of failing deep inside the seed with an opaque IndexError. Verified against a fresh database: migrations alone leave exactly one lexicon term, and the seed now loads all 1144 and completes.
Contributor
Coverage✅ 79.05% total — gate is 75%. Coverage for the Python files changed in this PR
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What broke
The development-mode startup seed (
core/app.pylifespan →seed_all) callsensure_seed_prereqs, which loadedcore/lexicon.jsononly whenlexicon_termwas completely empty:Since
b2c3d4e5f6a7_transducer_data_maturitythe migration inserts a lexicon term of its own. On a fresh database, migrations alone now leave exactly one row inlexicon_term— measured, not guessed — so that emptiness check reports "already loaded" and the real 1144-term lexicon is skipped entirely.The fallout, in order:
init_parameterviolatesparameter_default_unit_fkey, becauseftanddimensionlesswere never inserted as lexicon terms. Two parameters fail and the errors are printed, not raised.seed_allcallsrandom.choice(organization_terms)on an emptyorganizationcategory →IndexError: Cannot choose from an empty sequence.Application startup failed. Exiting.The frontend felt this as a CI outage: OcotilloUI's Cypress job polls
http://localhost:8000/docsfor the API, which was never going to answer, so the job burned its full 720s and failed with exit 124. See DataIntegrationGroup/OcotilloUI#350.The fix
ensure_seed_prereqsnow runsinit_lexicon()andinit_parameter()unconditionally. Both already leave existing rows alone —init_lexiconupserts categories, terms, and associations withon_conflict_do_nothing— so the emptiness shortcut was buying nothing and cost correctness.init_parameterskips(parameter_name, matrix)pairs that already exist, so a re-run no longer tripsuq_parameter_name_matrixand prints a wall of caughtDatabaseErrors.assert_lexicon_readyfails with the empty categories named, rather than leaving the next contributor to decode anIndexErrorthrown deep inside the seed.Verification
Against a fresh PostGIS database, recreating the CI sequence (drop schema →
alembic upgrade head→seed_all):lexicon_termrows after migrations only = 1, then the seed loads all 1144 and completes —contacts=2,SEED OK.staging's versions of the two files restored: the exact CI failure reproduces, same two FK violations onftanddimensionless, sameIndexError: Cannot choose from an empty sequence.Tests: 4 new cases in
tests/test_seed_prereqs.pycovering that both initializers run even when the tables hold rows, thatassert_lexicon_readynames the empty categories, that every required category has terms once reference data loads, and thatinit_parameteris idempotent.uv run pytest --ignore=tests/transfers→ 664 passed, 46 skipped, 6 xpassed. One unrelated failure,tests/test_ogc.py::test_ogc_project_areas_items_expose_groups_with_project_areas, reproduces identically on unmodifiedstagingon this machine and is green on CI runners — local environment (arm64 host, amd64 PostGIS image), not this change.blackand the pre-commitflake8selection are clean.🤖 Generated with Claude Code