Skip to content

fix(spec): the blueprint mirror the model generates against carries the applier's SNAKE_CASE constraint (cloud#1967) - #15798

Merged
hotlong merged 1 commit into
mainfrom
fix/blueprint-strict-schema-parity-1967
Sep 5, 2026
Merged

fix(spec): the blueprint mirror the model generates against carries the applier's SNAKE_CASE constraint (cloud#1967)#15798
hotlong merged 1 commit into
mainfrom
fix/blueprint-strict-schema-parity-1967

Conversation

@hotlong

@hotlong hotlong commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Part of objectstack-ai/cloud#1967 (cloud epic objectstack-ai/cloud#1955). The confirm-path half is a companion cloud PR; this is the producer-side half and reaches cloud only through a .objectstack-sha bump.

The two declarations disagreed about values

packages/spec/src/ai/solution-blueprint.zod.ts declares one shape twice:

  • SolutionBlueprintSchema — the authoring schema. Cloud's apply_blueprint validates the approved blueprint against it. Every identifier leaf carries .regex(/^[a-z_][a-z0-9_]*$/).
  • SolutionBlueprintStrictSchema — the OpenAI-strict structured-output contract the design model generates against. Not one identifier leaf carried it.

The strict mirror ↔ lenient schema — key parity test in this file already pins which keys each side carries, and its own comment records why: a key present on one side and missing on the other silently drops authored config. Nothing pinned the constraints on those keys, and 20 leaves had drifted.

What it cost, measured

A CRM build on the cloud golden-journey gate (run mtmgrsfbkusx, prompt 「帮我搭建一个 CRM:客户、联系人、商机…」): the design model wrote a company_size select whose option values came off their labels — 1_49 for 「1-49人」. Generating that was legal. On the turn the user clicked 「确认,开始搭建」, the deterministic confirm replay handed that exact blueprint to apply_blueprint, which refused it wholesale:

Blueprint failed validation — nothing was staged.
objects.0.fields.2.options.0.value: Invalid string: must match pattern /^[a-z_][a-z0-9_]*$/

Nothing was staged. The app existed at the end only because the model noticed the error card and re-applied a different, repaired plan the user had never seen. 1 of 4 measured builds — latent, model-word-choice dependent, not constant.

The change

Every identifier leaf in the mirror now reuses the same SNAKE_CASE regex through two shared builders (strictIdent / strictIdentOrNull), so the two declarations cannot drift on this axis by construction: object / field / view / dashboard / widget / app / nav names, reference, nameField, columns, groupBy, measure, roll-up object / field / relationshipField, condition field, and select option value. The regex is emitted into the JSON Schema the model is handed (pattern, on all 21 emitted string leaves — measured with z.toJSONSchema), so an out-of-pattern identifier is refused at generation rather than after the user has approved it.

Option value additionally states the rule that produced the incident: it may never start with a digit, so 「1-49人」 is authored as size_1_49. The label beside it is untouched and explicitly documented as free text in the user's own language — the constraint is on the stored identifier alone, never on what a person reads.

Why the pattern is safe under strict structured outputs. The mirror's own header comment enumerates exactly two strict-mode limits — every property present in required, and no open-ended additionalProperties — and neither is touched here. Two live structured-output schemas on the same cloud adapter and gateway already ship constraint keywords of this class (QueryPlanSchema's .min(1) / .int().min(1).max(200), JudgeOutputSchema's .min(0).max(100)), and cloud's sibling authoring tools already publish pattern: '^[a-z_][a-z0-9_]*$' on option values in their model-facing parameter schemas.

Tests

New strict mirror ↔ lenient schema — VALUE parity (cloud#1967) block, the value-side twin of the key-parity gate:

  • a leaf-by-leaf walk indexes every string leaf of both schemas by authoring path (wrapper nodes — optional / nullable / default / lazy — are transparent, so the lenient .optional() and the strict .nullable() spelling of one key land on the same path) and asserts they enforce the same pattern wherever both carry the key. It fails on any future divergence, in either direction.
  • the exact live value: 1_49 on a 「1-49人」 label is rejected by the applier's schema and by the mirror.

Both fail on the parent commit. Measured before the fix — the walk printed all 20 drifted leaves:

objects[].name · objects[].nameField · objects[].fields[].name · objects[].fields[].reference
objects[].fields[].options[].value · objects[].fields[].summaryOperations.object
objects[].fields[].summaryOperations.field · …relationshipField · …conditions[].field
views[].object · views[].name · views[].columns[] · views[].groupBy
dashboards[].name · dashboards[].widgets[].id · …object · …measure · …groupBy
app.name · app.nav[].target          (lenient /^[a-z_][a-z0-9_]*$/ vs strict none)

Verification

Run at 065ee300a:

  • pnpm --filter @objectstack/spec exec vitest run src/ai/ src/type-alias-convention.pin.test.ts9 files / 203 tests passed (2 of them failing on the parent commit, as above)
  • pnpm --filter @objectstack/spec typecheck — clean, including check:scripts-typecheck and check:test-typecheck
  • pnpm --filter @objectstack/spec build, then the build-reading gates: check:api-surface · check:browser-reachable-entries · check:dual-source-exports · check:entry-nameability · check:exported-any — all exit 0 (they refuse to answer against an unbuilt dist, so they were re-run after the build, not before)
  • source-reading gates: check:authorable-surface · check:docs · check:empty-state · check:export-origins · check:liveness · check:llms-txt · check:objectui-pin-citations · check:skill-refs · check:strictness-ledger · check:variant-docs · check:yaml-examples — all exit 0
  • pnpm check:nul-bytes · pnpm check:changeset-gate-self-tests · node scripts/check-adr-0087-registration.mjs --base origin/main — all exit 0 (the last re-run after committing the changeset: 1 non-breaking changeset(s) seen)

scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack derives 65 families for this surface. The set above is the spec-scoped subset plus the changeset and byte gates — a declared narrowing; the remaining families (self-tests of unrelated scripts, repo-wide censuses, sharded CI attestations, and the six the tool marks not runnable locally) are CI's run. The shared verify lock printed UNLOCKED (declared) — no usable flock on this host, so the shared verify lock was NEVER taken and NOTHING was serialized, so the wall-clock figures it reported are shared-box readings.

🤖 Generated with Claude Code

…he applier's SNAKE_CASE constraint

The strict structured-output mirror and the lenient authoring schema are two
declarations of one shape. An existing test pinned their KEYS; nothing pinned
the constraints on those keys, and 20 identifier leaves had drifted — every one
of them `.regex(SNAKE_CASE)` on the lenient side and unconstrained on the
mirror. A design model could therefore emit `1_49` (from a 「1-49人」 label) and
`apply_blueprint`, which validates against the lenient schema, refused the whole
blueprint on the turn the user approved it.

Every identifier leaf in the mirror now reuses the same regex, so the pattern
rides into the JSON Schema the model is given and an out-of-pattern identifier
is refused at generation rather than after approval. Option `value` states the
leading-digit rule explicitly (「1-49人」 → `size_1_49`); the `label` is
untouched, so only the stored value becomes an identifier.

A VALUE-parity test walks both schemas leaf by leaf, the twin of the key-parity
gate that already guards this pair.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation protocol:ai tests tooling labels Sep 5, 2026
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec, touching 18 documentable anchor(s).

26 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json fa85759963fcf25a55cd9a159c8e9c44bb9dde13.

4 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 1 anchor(s) matched too much of the corpus to be a work list: created_at (literal, 33 pages)
  • 5 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 129 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json fa85759963fcf25a55cd9a159c8e9c44bb9dde13packageMentionDocs.

Which tree this was computed on

This run read content/docs from a78b6e8d8d23de7f4dea8aa1bb8f9d213b335500 — the merge of head 065ee300a249373a6e9544b0637ff4ed0dbab8b5 into base fa85759963fcf25a55cd9a159c8e9c44bb9dde13, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin a78b6e8d8d23de7f4dea8aa1bb8f9d213b335500 && git checkout a78b6e8d8d23de7f4dea8aa1bb8f9d213b335500
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin fa85759963fcf25a55cd9a159c8e9c44bb9dde13 065ee300a249373a6e9544b0637ff4ed0dbab8b5 && git checkout -B drift-repro fa85759963fcf25a55cd9a159c8e9c44bb9dde13 && git merge --no-ff 065ee300a249373a6e9544b0637ff4ed0dbab8b5

node scripts/docs-audit/affected-docs.mjs --json fa85759963fcf25a55cd9a159c8e9c44bb9dde13

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs fa85759963fcf25a55cd9a159c8e9c44bb9dde13 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@hotlong
hotlong marked this pull request as ready for review September 5, 2026 09:00
@hotlong
hotlong added this pull request to the merge queue Sep 5, 2026
Merged via the queue into main with commit 3e3ecb0 Sep 5, 2026
36 checks passed
@hotlong
hotlong deleted the fix/blueprint-strict-schema-parity-1967 branch September 5, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:ai size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant