Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .changeset/blueprint-strict-mirror-value-parity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
"@objectstack/spec": minor
---

The model-facing solution-blueprint mirror can no longer generate an identifier the applier rejects.

`SolutionBlueprintSchema` (what `apply_blueprint` validates against) and `SolutionBlueprintStrictSchema` (the OpenAI-strict structured-output contract the design model generates against) are two declarations of one shape. Their KEYS were pinned by an existing parity test; their VALUES had never been. Every identifier in the lenient schema carried `.regex(/^[a-z_][a-z0-9_]*$/)` and not one identifier in the strict mirror carried it — 20 leaves apart, measured.

The consequence was a build whose approval did nothing. Asked for a CRM, the design model emitted a `company_size` select whose option values came straight off the labels — `1_49` for 「1-49人」. Generating that was legal. Applying it was not: on the turn the user clicked 「确认,开始搭建」 the deterministic confirm replay handed that exact blueprint to `apply_blueprint`, which refused it wholesale (`objects.0.fields.2.options.0.value: Invalid string: must match pattern /^[a-z_][a-z0-9_]*$/`) and staged nothing. The app appeared only because the model noticed the error card and retried with a repaired blueprint the user had never seen.

Every identifier leaf in the strict mirror now carries the same `SNAKE_CASE` constraint the lenient schema enforces — object / field / view / dashboard / widget / app / nav names, `reference`, `nameField`, `columns`, `groupBy`, `measure`, roll-up `object` / `field` / `relationshipField`, condition `field`, and select option `value`. The constraint is emitted into the JSON Schema the model is given (`pattern`), so an out-of-pattern identifier is refused at generation instead of after approval. Option `value` additionally spells out the case that produced the incident: it may never start with a digit, so 「1-49人」 is authored as `size_1_49` — the `label` keeps the human wording untouched, and only the stored value is an identifier.

A new `strict mirror ↔ lenient schema — VALUE parity` test walks both schemas leaf by leaf and fails on any future divergence, the value-side twin of the key-parity gate that already guards this pair.

Refs cloud#1967.
129 changes: 129 additions & 0 deletions packages/spec/src/ai/solution-blueprint.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -535,3 +535,132 @@ describe('strict mirror ↔ lenient schema — key parity', () => {
expect(parsed.objects[0].fields[2].expression).toBe("record.order_no + ' · ' + record.customer");
});
});

// ---------------------------------------------------------------------------
// VALUE parity — the twin of the key-parity gate above (cloud#1967).
//
// The key gate pins WHICH keys each side carries. Nothing pinned the
// CONSTRAINTS on those keys, and they had drifted: every identifier in the
// lenient schema carried `.regex(SNAKE_CASE)`, the strict mirror carried none.
// So the model could legally GENERATE `1_49` for a select option value (from a
// 「1-49人」 label) and `apply_blueprint` — which validates against the LENIENT
// schema — then rejected the very blueprint the user had approved:
// objects.0.fields.2.options.0.value: Invalid string: must match pattern
// /^[a-z_][a-z0-9_]*$/
// Nothing was staged, the approval became a no-op, and the build landed only
// because the model happened to retry with a repaired blueprint the user never
// saw. Two declarations of one contract, disagreeing about values.
// ---------------------------------------------------------------------------
describe('strict mirror ↔ lenient schema — VALUE parity (cloud#1967)', () => {
/** The regex a zod string leaf enforces, or null when it enforces none. */
const patternOf = (schema: any): string | null => {
const checks = schema?.def?.checks;
if (!Array.isArray(checks)) return null;
for (const c of checks) {
const p = c?._zod?.def?.pattern;
if (p) return String(p);
}
return null;
};

/**
* Index every string leaf reachable from `schema` by its authoring path,
* mapping it to the pattern it enforces. Wrapper nodes (optional / nullable /
* default / lazy) are transparent so the lenient `.optional()` and the strict
* `.nullable()` spelling of the same key land on the SAME path.
*/
const indexPatterns = (schema: any): Map<string, string | null> => {
const out = new Map<string, string | null>();
const seen = new Set<unknown>();
const walk = (node: any, path: string, depth: number): void => {
if (!node || depth > 12) return;
const def = node.def;
if (!def) return;
switch (def.type) {
case 'optional':
case 'nullable':
case 'default':
case 'prefault':
case 'nonoptional':
case 'readonly':
return walk(def.innerType, path, depth + 1);
case 'lazy':
return walk(def.getter(), path, depth + 1);
case 'array':
return walk(def.element, `${path}[]`, depth + 1);
case 'object': {
if (seen.has(def.shape)) return;
seen.add(def.shape);
for (const [key, child] of Object.entries(def.shape as Record<string, unknown>)) {
walk(child, path ? `${path}.${key}` : key, depth + 1);
}
return;
}
case 'string':
out.set(path, patternOf(node));
return;
default:
return; // unions / enums / records / numbers carry no identifier pattern
}
};
walk(schema, '', 0);
return out;
};

it('every identifier the applier constrains is constrained the same way in the model-facing mirror', () => {
const lenient = indexPatterns(SolutionBlueprintSchema);
const strict = indexPatterns(SolutionBlueprintStrictSchema);
// Only paths BOTH sides carry are comparable — the key-parity gate above
// owns "which keys exist"; this one owns "what values they accept".
const drift = [...lenient.entries()]
.filter(([path]) => strict.has(path))
.filter(([path, pattern]) => strict.get(path) !== pattern)
.map(([path, pattern]) => `${path}: lenient ${pattern ?? 'none'} vs strict ${strict.get(path) ?? 'none'}`);
expect(drift).toEqual([]);
});

it('the model cannot emit the leading-digit option value the applier rejects', () => {
// The exact value from the live run: a 「1-49人」 company-size band.
const optionsWithLeadingDigit = [
{ label: '1-49人', value: '1_49' },
{ label: '50-199人', value: '50_199' },
];
const lenient = SolutionBlueprintSchema.safeParse({
summary: 's',
objects: [{
name: 'customer',
fields: [{ name: 'company_size', type: 'select', options: optionsWithLeadingDigit }],
}],
});
expect(lenient.success).toBe(false);

const strict = SolutionBlueprintStrictSchema.safeParse({
summary: 's',
assumptions: [],
questions: null,
objects: [{
name: 'customer',
label: null,
description: null,
sharingModel: null,
nameField: null,
fields: [{
name: 'company_size',
label: null,
type: 'select',
required: null,
reference: null,
options: optionsWithLeadingDigit,
summaryOperations: null,
expression: null,
}],
}],
views: null,
dashboards: null,
app: null,
});
// Before cloud#1967 this parse SUCCEEDED — the proposal was legal to
// generate and illegal to apply.
expect(strict.success).toBe(false);
});
});
62 changes: 38 additions & 24 deletions packages/spec/src/ai/solution-blueprint.zod.ts
Original file line number Diff line number Diff line change
Expand Up @@ -256,30 +256,47 @@ export function defineSolutionBlueprint(config: z.input<typeof SolutionBlueprint
// SolutionBlueprintSchema} (and every existing consumer/test) is unchanged.
// ---------------------------------------------------------------------------

// Identifier leaves in this mirror carry the SAME `SNAKE_CASE` constraint the
// lenient schema above enforces (cloud#1967). They had drifted: the mirror is
// what the model may GENERATE, the lenient schema is what `apply_blueprint`
// VALIDATES, so an identifier legal here and illegal there produced a proposal
// the applier rejected wholesale — `1_49` from a 「1-49人」 label, staging
// nothing on the turn the user clicked 「确认,开始搭建」. Declaring the constraint
// on the generating side makes that value impossible to propose rather than
// impossible to apply. The `strict mirror ↔ lenient schema — VALUE parity` test
// walks both schemas leaf by leaf and fails on any future divergence.
const strictIdent = (description: string) => z.string().regex(SNAKE_CASE).describe(description);
/** The same identifier, `.nullable()` — this mirror's spelling of "optional". */
const strictIdentOrNull = (description: string) =>
z.string().regex(SNAKE_CASE).nullable().describe(description);

// The roll-up config, strict-shaped: every key present, "optional" → nullable,
// and the predicate as a flat `conditions` ARRAY because strict mode cannot
// express the canonical `filter` map (open-ended additionalProperties). The
// blueprint tools compile `conditions` back into a real query filter.
const StrictSummaryOperations = z.object({
object: z.string().describe('The CHILD object whose records are aggregated (snake_case). It MUST have a lookup/master_detail field pointing back at this parent.'),
object: strictIdent('The CHILD object whose records are aggregated (snake_case). It MUST have a lookup/master_detail field pointing back at this parent.'),
function: z.enum(['count', 'sum', 'avg', 'min', 'max']).describe('Aggregation: "数量/个数/计数" → count; "合计/总额/累计" → sum; "平均" → avg'),
field: z.string().nullable().describe('Numeric field on the CHILD to aggregate; null (or "id") for count'),
relationshipField: z.string().nullable().describe('Child FK field back to this parent, or null to auto-detect'),
field: strictIdentOrNull('Numeric field on the CHILD to aggregate; null (or "id") for count'),
relationshipField: strictIdentOrNull('Child FK field back to this parent, or null to auto-detect'),
conditions: z.array(z.object({
field: z.string().describe('Field on the CHILD object'),
field: strictIdent('Field on the CHILD object'),
op: z.enum(['lt', 'lte', 'gt', 'gte', 'eq', 'ne']).describe('Comparison operator'),
value: z.union([z.number(), z.string(), z.boolean()]).describe('Comparison value — a select field\'s option VALUE, never its label'),
})).nullable()
.describe('CONDITIONAL roll-up: aggregate only child rows matching these (ANDed), or null to aggregate every child. REQUIRED whenever the field name carries a qualifier ("已完成任务数 / 已收货金额 / 待处理工单数", any 已X / 未X / <某状态>的 count-or-sum) — e.g. [{field:"status",op:"eq",value:"completed"}]. Without it the roll-up counts EVERYTHING and reports a plausible WRONG number.'),
});

const StrictField = z.object({
name: z.string().describe('Field machine name (snake_case)'),
name: strictIdent('Field machine name (snake_case)'),
label: z.string().nullable().describe('Human-readable field label, or null'),
type: FieldType.describe('Field data type'),
required: z.boolean().nullable().describe('Whether the field is required, or null'),
reference: z.string().nullable().describe('Target object for lookup/master_detail, or null'),
options: z.array(z.object({ label: z.string(), value: z.string() })).nullable()
reference: strictIdentOrNull('Target object for lookup/master_detail, or null'),
options: z.array(z.object({
label: z.string().describe('What the user reads on the dropdown — free text in the user\'s own language (「1-49人」, 「已完成」).'),
value: strictIdent('The STORED machine value: snake_case, and it may NEVER start with a digit — give it a word prefix instead (「1-49人」 → "size_1_49", 「2024年」 → "year_2024"). The label carries the human wording; this key only has to be a legal identifier.'),
})).nullable()
.describe('Choices for select-family fields, or null'),
summaryOperations: StrictSummaryOperations.nullable()
.describe('REQUIRED when type is "summary" (a roll-up of child records onto this parent: 任务总数 / 报名人数 / 合计金额 / 已完成任务数); null for every other field type. A "summary" field without it is runtime-dead — it reads 0/empty everywhere.'),
Expand All @@ -288,39 +305,36 @@ const StrictField = z.object({
});

const StrictObject = z.object({
name: z.string().describe('Object machine name (snake_case)'),
name: strictIdent('Object machine name (snake_case)'),
label: z.string().nullable().describe('Human-readable singular label, or null'),
description: z.string().nullable().describe('What this object represents, or null'),
fields: z.array(StrictField).describe('Fields to create on the object'),
sharingModel: z.enum(['private', 'public_read', 'public_read_write', 'controlled_by_parent']).nullable()
.describe('Org-Wide Default record visibility (OWD) for INTERNAL users (ADR-0090), or null to accept the platform default (business object → public_read_write; master-detail child → controlled_by_parent). SET it when the user\'s description implies a visibility intent: personal/private data (HR, 绩效, salary, 个人隐私) → "private" (owner-only); "public_read" = everyone reads, owner writes; "public_read_write" = everyone reads+writes; "controlled_by_parent" ONLY for an object with a master_detail reference field. Null on privacy-sensitive data silently over-shares it.'),
nameField: z.string().nullable()
.describe('The record title field — which field holds the human-readable name shown on cards, lookup chips, breadcrumbs and search (ADR-0079), or null to let the platform auto-pick a text field. Set it to the object\'s text label field (e.g. "product_name") — snake_case. For a numbered entity (invoice/ticket), set it to a formula field that composes number + name (e.g. "{order_no} · {customer}"). Declaring it is strongly preferred over null.'),
nameField: strictIdentOrNull('The record title field — which field holds the human-readable name shown on cards, lookup chips, breadcrumbs and search (ADR-0079), or null to let the platform auto-pick a text field. Set it to the object\'s text label field (e.g. "product_name") — snake_case. For a numbered entity (invoice/ticket), set it to a formula field that composes number + name (e.g. "{order_no} · {customer}"). Declaring it is strongly preferred over null.'),
});

const StrictView = z.object({
object: z.string().describe('Object this view displays (snake_case)'),
name: z.string().describe('View machine name (snake_case)'),
object: strictIdent('Object this view displays (snake_case)'),
name: strictIdent('View machine name (snake_case)'),
label: z.string().nullable().describe('Human-readable view label, or null'),
type: z.enum(['list', 'form', 'kanban', 'calendar', 'gallery', 'gantt']).nullable().describe('View kind, or null for list. "gallery" = visual card/cover browse (画廊/相册/卡片墙/封面/海报, or an object with an image/avatar/file field); "gantt" = timeline/schedule (甘特图/时间线/排期, object with BOTH a start and an end date field); "kanban" = board grouped by a status/select field; "calendar" = single-date schedule; "form" = record editor.'),
columns: z.array(z.string()).nullable().describe('Field names shown as columns, or null. For a gallery, INCLUDE the image/avatar/file field (becomes the card cover); for a gantt, INCLUDE the start date column before the end date column.'),
groupBy: z.string().nullable().describe('REQUIRED for kanban: the select/status field whose options become the board columns (e.g. "stage"). Optional for gantt (groups leaf tasks). Null for list/form/calendar/gallery.'),
columns: z.array(z.string().regex(SNAKE_CASE)).nullable().describe('Field names shown as columns, or null. For a gallery, INCLUDE the image/avatar/file field (becomes the card cover); for a gantt, INCLUDE the start date column before the end date column.'),
groupBy: strictIdentOrNull('REQUIRED for kanban: the select/status field whose options become the board columns (e.g. "stage"). Optional for gantt (groups leaf tasks). Null for list/form/calendar/gallery.'),
});

const StrictDashboard = z.object({
name: z.string().describe('Dashboard machine name (snake_case)'),
name: strictIdent('Dashboard machine name (snake_case)'),
label: z.string().nullable().describe('Human-readable dashboard label, or null'),
widgets: z.array(z.object({
id: z.string().describe('Widget id (snake_case)'),
id: strictIdent('Widget id (snake_case)'),
title: z.string().nullable().describe('Widget title, or null'),
object: z.string().nullable().describe('Source object, or null'),
object: strictIdentOrNull('Source object, or null'),
chart: z.enum(['metric', 'bar', 'line', 'pie', 'table']).nullable().describe('Visualization, or null'),
measure: z.string().nullable()
.describe('The field this widget aggregates (e.g. "amount", "probability"), or "count" to count records, or null to infer from the title. The aggregation (sum vs average) is chosen automatically from the field type — name the FIELD, not "total_amount". "total revenue" → "amount"; "average win rate" → "win_rate"; "number of deals" → "count".'),
groupBy: z.string().nullable()
.describe('The field to break the widget down by — the category or time axis (e.g. "stage", "created_at"), or null for a single-number metric. A "by status" chart MUST set this to the status field; the title and this field MUST name the SAME field.'),
measure: strictIdentOrNull('The field this widget aggregates (e.g. "amount", "probability"), or "count" to count records, or null to infer from the title. The aggregation (sum vs average) is chosen automatically from the field type — name the FIELD, not "total_amount". "total revenue" → "amount"; "average win rate" → "win_rate"; "number of deals" → "count".'),
groupBy: strictIdentOrNull('The field to break the widget down by — the category or time axis (e.g. "stage", "created_at"), or null for a single-number metric. A "by status" chart MUST set this to the status field; the title and this field MUST name the SAME field.'),
condition: z.object({
field: z.string().describe('Field on the widget object to filter by (e.g. "stock_quantity", "status")'),
field: strictIdent('Field on the widget object to filter by (e.g. "stock_quantity", "status")'),
op: z.enum(['lt', 'lte', 'gt', 'gte', 'eq', 'ne']).describe('Comparison operator'),
value: z.union([z.number(), z.string(), z.boolean()]).describe('Comparison value (e.g. 10, "open")'),
}).nullable()
Expand All @@ -330,13 +344,13 @@ const StrictDashboard = z.object({

const StrictNavItem = z.object({
type: z.enum(['object', 'dashboard']).describe('What this nav entry opens'),
target: z.string().describe('Object or dashboard machine name to surface (snake_case)'),
target: strictIdent('Object or dashboard machine name to surface (snake_case)'),
label: z.string().nullable().describe('Nav entry label, or null'),
icon: z.string().nullable().describe('Lucide icon name, or null'),
});

const StrictApp = z.object({
name: z.string().describe('App machine name (snake_case)'),
name: strictIdent('App machine name (snake_case)'),
label: z.string().nullable().describe('App display label, or null'),
icon: z.string().nullable().describe('Lucide icon for the App Launcher, or null'),
nav: z.array(StrictNavItem).nullable()
Expand Down
Loading