Translating a taxonomy silently resets `collections`/`hierarchical` — identity-level fields have no owner in the row-per-locale model

[MA2153]

Description

Translating a taxonomy silently resets the fields that describe it. Creating the Spanish translation exactly the way the API documents it — POST /_emdash/api/taxonomies with translationOf and a translated label — leaves the es def declaring collections: [] and hierarchical: false, because handleTaxonomyCreate defaults both (handlers/taxonomies.ts:322, :370) and never inherits them from the source. getCollectionTaxonomyNames then filters defs by collections on the locale-resolved row, so the taxonomy applies to posts in en and not in es, and a hierarchical taxonomy comes out flat in translation.

No unusual input is required. Omitting fields that describe the taxonomy rather than one locale's presentation of it is the happy path — there is nothing to send, and the defaults win.

Separately, one call to the MCP menu_set_items tool NULLs translation_group on every item in the menu. MenuRepository.setItems is the only insert path that never writes the column (menu.ts:601-615); createItem, the menu-clone path and seed/apply.ts:1137 all set it. Nothing reads it at runtime, so nothing looks wrong — until emdash export-seed (export-seed.ts:594 reads it to emit translationOf), at which point export → re-import mints a fresh group per item and the en/es correspondence is gone.

Both are the same root cause, and it is not confined to these two call sites: the row-per-locale + translation_group model was copied from content entries (019) to six system tables, and on those six nothing owns the fields that identify the thing. Details in Root cause below.

Verified

Against main @ 14017995 (packages/core 0.33.0) with an integration test driving the real handlers on SQLite:

• A — en def: collections = ["post"], hierarchical = 1. Its es translation, same translation_group: collections = "[]", hierarchical = 0.
• B — after setItems with two items, both rows have translation_group = NULL.
• C — with locales: ["ar", "en"], translating a hierarchical en taxonomy to ar leaves ar at hierarchical = 0. requireTaxonomyDef (handlers/taxonomies.ts:146) resolves without a locale using orderBy("locale", "asc") + executeTakeFirst, so the ar row wins and handleTermList returns the tree flattened — a parent and its child both come back top-level — for a request at locale=en. Control run without the ar translation returns the correct nesting.

The model is dialect-independent; D1/Workers behaves the same.

Steps to reproduce

A — structural drift on the happy path (no divergent input required)

1. Configure locales: ["en", "es"].
2. POST /_emdash/api/taxonomies → { "name": "topic", "label": "Topics", "hierarchical": true, "collections": ["posts"] }
3. Create the Spanish translation the way the API intends:
   POST /_emdash/api/taxonomies → { "name": "topic", "label": "Temas", "locale": "es", "translationOf": "<id from step 2>" }
4. SELECT locale, collections, hierarchical FROM _emdash_taxonomy_defs WHERE name = 'topic' → en keeps ["posts"]/1, es gets []/0, both on the same translation_group.
5. getEmDashCollection("posts", { locale: "es" }) → entries carry no topic terms. The same call at en does.

Expected: collections and hierarchical describe the taxonomy, not one locale's presentation of it, so they should be inherited (or shared) rather than silently reset. The same applies to PUT /_emdash/api/taxonomies/:name?locale=xx, which can set either field on one locale only.

B — menu item translation linkage destroyed

1. Two locales configured; menu primary with items in en, translated to es via POST /_emdash/api/menus with translationOf (items are cloned with their groups intact).
2. Call the MCP menu_set_items tool on primary.
3. SELECT id, translation_group FROM _emdash_menu_items WHERE menu_id = '<primary en>' → every translation_group is NULL.
4. emdash export-seed → no translationOf on any item; re-importing mints a fresh group per item, so the en/es correspondence is gone.

Expected: setItems writes translation_group like every other insert path.

Fallouts

#	Surface	Behaviour
1	Taxonomy scope + hierarchy — handlers/taxonomies.ts:322, :370, :444	Translation-create resets collections/hierarchical; update writes them to one locale's row. The handleTaxonomyUpdate docblock claims "every field here belongs to a single locale's definition", which isn't true for those two. taxonomies/index.ts:390-393 already documents the drift ("per-locale rows of the same def can drift in their declared collections") and works around it by folding the collection scope into a cache key.
1b	Term tree collapses — handlers/taxonomies.ts:146, :608	handleTermList reads hierarchical from requireTaxonomyDef(name) with no locale, which takes the alphabetically first locale's row. Once a drifted translation exists at a locale code sorting before the source's (ar before en), every GET /_emdash/api/taxonomies/{name}/terms returns the tree flat, at every locale — including the source. Admin-UI-reachable: TaxonomySidebar term picker (fetchTerms, TaxonomySidebar.tsx:86). Verified as C above.
2	Machine names diverge — handleTaxonomyCreate, handleMenuCreate	Both accept input.name freely alongside translationOf, with no equality check against the source. Terms are keyed on taxonomies.name, so an es def named categoria sharing a group with en's category orphans every es term. Menus are fetched by findByName(name, locale), so a divergent name makes the translation unreachable.
3	Menu item identity — menu.ts:601-615, export-seed.ts:594	setItems inserts with translation_group unset (nullable, no default). Invisible at runtime; destroys cross-locale item identity on the next export → import round-trip.
4	Junction tables disagree on what they key — content_taxonomies.entry_id, _emdash_content_references.parent_group/child_group vs _emdash_content_bylines.content_id	The first two store the entry's translation_group, so one assignment covers every locale. The third stores a row id and is duplicated per translation by copyContentBylines. On the same post, editing categories affects every locale while editing credits affects one.
5	Group-level assignment, per-locale render — taxonomy.ts:550, migration 040	Pivots are locale-agnostic; hydration is strict per locale by design. getTermsForEntry drops terms "whose translation_group lacks a row in the requested locale", and 040 states the same for bylines ("a credit at locale X renders iff a byline row exists at locale X… There is no read-time fallback"). An untranslated term simply vanishes from the es site, with nothing in the data distinguishing that from "deliberately uncategorised". #2508 (open) narrows this: public hydration falls back preferred → default locale inside the folded query, and the admin terms route returns availableLocales with an inline create-translation action. The vanish case shrinks to terms present in neither locale, and the underlying question — what a locale-neutral assignment means at render time — is answered by policy rather than by the model.

Reachability, for triage: 1 and 2 are REST/MCP/SDK surfaces (the admin only sends translationOf for content); 3 is MCP-only; 1b, 4 and 5 are reachable from the admin UI with no unusual steps.

Root cause: no owner for identity-level fields

The model was copied from content entries (019) to six system tables: taxonomies, _emdash_taxonomy_defs, _emdash_menus, _emdash_menu_items (036), _emdash_bylines (040), and _emdash_relations (043).

For content it holds: every column of an ec_* row is per-locale, and translation_group is pure linkage. For these six it doesn't. Most of their columns describe the identity — _emdash_taxonomy_defs.hierarchical/collections, _emdash_menus.name, _emdash_menu_items.type/reference_id, _emdash_bylines.user_id — and only one or two are genuinely translatable (label, bio). Those identity columns are stored once per locale row with nothing keeping the copies in agreement.

Nothing enforces the invariant. UNIQUE(name, locale), UNIQUE(slug, locale) and (translation_group, locale) all constrain the row, never the group. There is no check, no reconciliation pass, and no admin surface that shows two locales disagreeing.

Three different mitigations already exist, each invented once:

• fan-out on write — TaxonomyRepository.update (parent_id, sort_order) and applyPositions. The only group-scoped UPDATEs in the codebase.
• immutable structural fields — _emdash_relations; relation.ts:39-40 records the rest as "a cross-group operation deferred to a later slice".
• a group-keyed side table — _emdash_byline_field_group_values plus a per-field translatable flag (042).

Everywhere else, writes are row-scoped and drift silently.

The decision this needs

Fallouts 1, 4 and 5 can't be fixed at the call site — patching handleTaxonomyCreate to inherit collections picks an answer to "is the entity the row or the group?" without stating it, and the next feature re-litigates it. That has already happened three times: 056 invented group fan-out for sort_order, 043 declared structural fields immutable and deferred the question, 042 built a second value table rather than answer it for the base columns. Every new column on these six tables is a fresh judgment call with no stated rule and no test that can catch getting it wrong.

Option 1 — the row is the entity (status quo, made explicit). Document that each locale owns its own structural fields; fix create to inherit from the source so the defaults stop winning; add a reconciliation view. Cheapest, no migration, but drift stays reachable and fallout 5 keeps its ambiguity.

Option 2 — the group is the entity, enforced by fan-out on write. Extend the existing TaxonomyRepository.update pattern to every identity column on all six tables. No schema change, but correctness depends on every writer remembering, and no constraint catches a missed one — which is how fallout 3 happened.

Option 3 — the group is the entity, enforced by storage. Per-field translatable flag plus a group-keyed table, as 042 already does for byline fields, extended to the base columns. Structurally prevents drift and collapses the per-locale read fallback (below), but it's the largest migration.

Two things any option has to answer:

1. What happens to data that has already drifted? Migrations are forward-only, and divergence is unrepairable after the fact: translation_group = id is a backfill convention, not a constraint, and the anchor row is freely deletable (TaxonomyRepository.delete:468, BylineRepository.delete:866, MenuRepository.delete:403 each drop one row and only cascade when it is the last sibling). Once it's gone there's no principled way to decide which surviving locale held the intended value.
2. Does it fix the logged-out read cost? Today, finding "the right row" is a search — one sequential query per locale down the fallback chain (menus/index.ts:71, taxonomies/index.ts:238, :258, :453, bylines/index.ts:102), and the common case (locale requested, translation absent) walks the whole chain before finding anything. Caching multiplies rather than amortises: getMenu keys on ${name}:${locale}, getTaxonomyDefs on defs:${localeKey}, so N locales hold N entries and pay N cold misses for structural data meant to be identical. Per CLAUDE.md the logged-out hot path only ratchets down, so option 3 is the only one that improves it; options 1 and 2 leave it as-is. fix: preserve taxonomy assignments across locale fallbacks #2508 (open) folds the entry-terms fallback into the existing query with logged-out counts unchanged, which settles this for that one path; the menu and byline chains and the per-locale cache keys are untouched.

Options 2 and 3 are breaking for anyone relying on per-locale structural fields today, so they'd need a package bump and a changeset calling the break out.

Fixable now, independent of that decision

Fallouts 2 and 3 are small, self-contained and unambiguous — nothing about the row-vs-group question changes what they should do:

• 2 — reject name mismatching the source when translationOf is present (or ignore it and copy the source's), in handleTaxonomyCreate and handleMenuCreate.
• 3 — setItems writes translation_group like every other insert path.

Environment

• emdash version: 0.33.0 (packages/core, main @ 14017995)
• Node.js version: v26.1.0
• Runtime: both (SQLite/Node and D1/Workers — the model is dialect-independent)
• OS: Linux (WSL2)

[ascorbic]

Thanks fore putting this together. I agree that these are real bugs, but I’d separate the immediate correctness fixes from the larger storage decision. These can be fixed now, independently:

• Translated taxonomies should inherit shared fields and reject a different machine name.
• Term listing should use the definition for the requested locale, not whichever locale sorts first.
• MenuRepository.setItems() must preserve or create each item’s translation_group.

These should go into the 1.0 milestone.

For the wider question, I don’t think every field can follow one blanket “row or group” rule. Labels and descriptions are translated, while things such as taxonomy names, hierarchy and collection scope describe the underlying thing and should remain consistent across translations. We need the equivalent decision for menus, menu items, bylines, terms and relations.

I think before 1.0 we need to investiogate if there's a way to safely migrate to an option 3-type, group-owned model. We need a proper decision on the schema for each entity (not just copy bylines if that model doesnt make sense in each case). To do this, we first need to work out (and document) the ownership for each entity (shared or per-locale) and the behaviour when updating. From that, we'd need to work out the migraiton path. It's unlikely this is something that could be done in a normal migration as there is likely to already be drift, and decisoons that need to be made.