diff --git a/.changeset/olive-eyes-hug.md b/.changeset/olive-eyes-hug.md new file mode 100644 index 0000000000..c643ea2151 --- /dev/null +++ b/.changeset/olive-eyes-hug.md @@ -0,0 +1,30 @@ +--- +'@objectstack/cli': patch +--- + +`os dev` over a host config now has ONE registrar for stack-declared security metadata + +A HOST config — one whose `plugins[]` holds instantiated plugins — skips +`createStandaloneStack`, so the composition that already declares +`securityMetadataRegistrar: 'artifact-door'` never runs. `os serve` then wrapped the +config module in `new AppPlugin(config)` under the default `'app-plugin'` registrar, +and under `os dev` it ALSO composed the dev-only HMR `MetadataPlugin` over +`dist/objectstack.json` — the compiled twin of that same module, which the `os dev` +supervisor had just produced. Both writers registered `positions`, `permissions`, +`capabilities` and `sharingRules` into the metadata service, from two sources of one +stack. + +The two copies did not differ by parsing — `defineStack()` is strict by default and +runs the same schema parse the artifact door runs, so both carry the schema defaults. +They differed by ADR-0010 provenance, and by freshness: the door re-ingests its copy +on every recompile while the module copy never refreshes. Measured on a real `os dev` +boot, the wrap registered last, so its copy won the cold boot — and the door's copy +replaced it on the first artifact reload, so which copy a consumer read changed +mid-run, with no restart and no signal. + +The `os dev` composition now declares `securityMetadataRegistrar: 'artifact-door'` on +that wrap exactly when it composes the HMR door over a compiled artifact that is +present on disk, so the door is the single registrar on this boot shape too. Nothing +changes when no door composes — `os serve`, `os migrate`, a host config whose artifact +has not been compiled or was named but is missing, and every production boot keep the +default `'app-plugin'` registrar and their only writer. diff --git a/packages/cli/src/commands/serve-host-config-security-registrar.pin.test.ts b/packages/cli/src/commands/serve-host-config-security-registrar.pin.test.ts new file mode 100644 index 0000000000..71eeeac773 --- /dev/null +++ b/packages/cli/src/commands/serve-host-config-security-registrar.pin.test.ts @@ -0,0 +1,207 @@ +// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Pin: **on a HOST config, `os dev` has exactly ONE registrar for the four + * ADR-0057 security collections — the artifact door — and only when that door + * actually composes.** + * + * ## The two writers this refuses, as measured + * + * `shouldBootWithLibrary()` returns `false` for a host config (one whose + * `plugins[]` holds instantiated plugins), so `createStandaloneStack` — the + * composition that already declares `securityMetadataRegistrar: + * 'artifact-door'` — never runs. Two other writers then reach the metadata + * service over the SAME stack: + * + * 1. `new AppPlugin(config)` wrapping the config MODULE. Under the default + * `'app-plugin'` registrar its ADR-0057 block registers `positions` / + * `permissions` / `capabilities` / `sharingRules`. + * 2. the dev-only HMR `MetadataPlugin`, over `dist/objectstack.json` — the + * COMPILED TWIN of that same module, which the `os dev` supervisor + * produced moments earlier. It strict-parses, forward-converts and + * ADR-0010-stamps, and reaches all four collections too. + * + * Both were measured on a real `os dev` boot of a host config (`examples/ + * app-showcase`, whose `plugins[]` holds four connector plugins and whose + * stack declares all four collections): + * + * → Compiling objectstack.config.ts → dist/objectstack.json... + * INFO [MetadataPlugin] Loading metadata from local artifact file + * {"path":".../examples/app-showcase/dist/objectstack.json"} + * INFO [MetadataPlugin] Artifact metadata loaded {...,"totalRegistered":246} + * INFO Registered stack-declared security metadata + * {"appId":"com.example.showcase","count":23} + * + * — the door at `21.215`, the wrap at `21.835`. `registerInMemory` is a + * `Map.set`, so **the wrap's copy wins the cold boot**; and because the door + * re-ingests on every artifact reload while the module copy never refreshes, + * **the winner changes mid-run**. Measured on an instrumented host config + * whose compiled twin carried a distinguishing label: the cold-boot registry + * held the module's labels and no `_packageVersion`, and 37s later — after one + * artifact reload, no restart — the same four items held the artifact's labels + * and `_packageVersion: '1.0.0'`. + * + * ⚠️ **The two copies differ by PROVENANCE and FRESHNESS, not by parsing.** On + * a config boot `defineStack()` is strict by default and runs the same + * `ObjectStackDefinitionSchema` parse the door runs (`packages/spec/src/ + * stack.zod.ts`), so the wrap's copy already carries the schema defaults and + * the ADR-0122 input transforms. What it lacks is the ADR-0010 stamp + * (`_packageVersion` on all four kinds, `_packageId` / `_provenance` on + * `position`), and — the half that bites — it never refreshes, while the + * door's copy reloads on every recompile. A consumer therefore reads one of + * two copies of an authorization input depending on when it asked. That is + * why this is `security`-labelled, and why the fix is the ownership one + * rather than "make the two shapes match". + * + * ## What is pinned, and why the guard is on SOURCE + * + * The decision lives inside `Serve.run()`, ~900 lines into a method that boots + * a kernel, a database and an HTTP server; there is no seam to call. The + * repo's answer for exactly this shape is a source pin + * (`child-env-source-loader.pin.test.ts`, `serve-settings-ordering.pin.test.ts`) + * — assert the STRUCTURE that makes the composition correct, and pair it with + * a behavioural assertion that the words the structure uses still mean + * something. Both halves are here: without the second, renaming the option on + * `AppPlugin` would leave this file green over a dead string. + * + * The structural invariant has two directions and both matter: + * + * • the wrap declares `'artifact-door'` when the door composes, and + * • it declares NOTHING (so `AppPlugin` defaults to `'app-plugin'`) when the + * door does not — because a host config with no compiled artifact, and + * every non-dev host boot, would otherwise lose its ONLY registrar. + * Measured: `os serve` over the same host config has a metadata service + * and the wrap is its only writer (`Registered stack-declared security + * metadata {"appId":"com.probe.hostcfg","count":4}`, no door in the boot). + * + * ⛔ The second direction has a trap that RESOLVING hides: under `os dev` the + * supervisor always writes its channel, and `resolveDefaultArtifactPath` + * returns an explicitly named path VERBATIM with no existence check + * (`packages/runtime/src/default-host.ts`) — only the conventional + * `/dist/objectstack.json` fallback is stat'ed. Compose the door over a + * path that is not on disk (`os dev --artifact ./typo.json`, a stale + * `OS_ARTIFACT_PATH`) and it starts EMPTY and SILENT: its local-file load is + * `{ optional: true }` and answers ENOENT with an `info` line, registering + * nothing (`packages/metadata/src/plugin.ts`). The wrap would have deferred to + * a writer that never writes, and all four collections would end the boot with + * ZERO registrars — green and quiet, and strictly worse than the divergence + * this composition removes. So the gate is EXISTENCE, not resolution. + * + * That is why the door instance is constructed next to the wrap and only + * `kernel.use`d at its ordering-constrained site: ONE value decides both + * facts. Two independent expressions would be free to drift, and the drift is + * invisible — a boot with no registrar looks exactly like a boot with one. + */ + +import { describe, it, expect } from 'vitest'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { AppPlugin } from '@objectstack/runtime'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const SERVE_TS = path.join(HERE, 'serve.ts'); +const source = fs.readFileSync(SERVE_TS, 'utf8'); + +/** Occurrences of `needle` in the source, as a plain substring count. */ +function count(needle: string): number { + return source.split(needle).length - 1; +} + +describe('#14397 — `os dev` over a HOST config composes ONE registrar for stack-declared security metadata', () => { + it('the dev artifact door is decided ONCE, before the AppPlugin wrap', () => { + expect(source, 'the door decision must be a single named value').toContain( + 'let devArtifactDoor: any;', + ); + // The gate is the same one the composition has always used. + expect(source).toContain( + "if (isDev && flags.server && !plugins.some((p: any) => p?.constructor?.name === 'MetadataPlugin')) {", + ); + // Exactly one MetadataPlugin is constructed in this file, and it is + // that value — a second construction site is a second decision. + expect(count('new MetadataPlugin(')).toBe(1); + expect(source).toContain('devArtifactDoor = new MetadataPlugin({'); + // The path still comes from the supervisor's own channel, never from + // `/dist/objectstack.json` by accident. + expect(source).toContain( + 'const hmrArtifactPath = resolveDefaultArtifactPath(readInternalArtifactPath());', + ); + }); + + it('the door is composed only when its artifact EXISTS, not merely resolves', () => { + // The regression this closes: `resolveDefaultArtifactPath` returns a + // NAMED path verbatim without stat'ing it, and the door tolerates + // ENOENT by starting empty — so gating on resolution alone hands the + // four collections to a writer that never writes. + expect(source, 'the door must be gated on the artifact being on disk').toContain( + 'if (!fs.existsSync(hmrArtifactPath)) {', + ); + // The gate must sit BEFORE the construction, not after it: a door + // constructed and then discarded would still have set the wrap's + // option under any future refactor that reads "was one built?". + const gateAt = source.indexOf('if (!fs.existsSync(hmrArtifactPath)) {'); + const buildAt = source.indexOf('devArtifactDoor = new MetadataPlugin({'); + expect(gateAt).toBeGreaterThan(-1); + expect(buildAt).toBeGreaterThan(-1); + expect(gateAt).toBeLessThan(buildAt); + // A missing artifact is not a silent downgrade: the warning names the + // path, and says the wrap keeps the collections. + expect(source).toContain( + '` ⚠ Dev metadata-HMR endpoint not enabled: no compiled artifact at ${hmrArtifactPath}`', + ); + expect(source).toContain('Stack-declared security metadata stays with the app wrap'); + }); + + it('the host-config wrap declares `artifact-door` exactly when that door exists', () => { + expect(source).toContain( + "devArtifactDoor ? { securityMetadataRegistrar: 'artifact-door' } : {},", + ); + // ⛔ The unconditional shape is the defect: it is what put a SECOND + // writer on every `os dev` boot of a host config — a copy that lacks + // the ADR-0010 provenance stamp and never refreshes, alongside the + // door's, which reloads on every recompile. + expect( + source, + 'the wrap must never be constructed without the registrar decision', + ).not.toContain('new AppPlugin(config)]'); + }); + + it('the door is `kernel.use`d from that same value, at its ordering-constrained site', () => { + expect(count('await kernel.use(devArtifactDoor);')).toBe(1); + expect(source).toContain('if (devArtifactDoor) {\n try {\n await kernel.use(devArtifactDoor);'); + // The `kernel.use` still sits AFTER the HonoServer composition — + // MetadataPlugin.start() mounts its route on the `http-server` + // service. Positions, not line numbers: the file moves. + expect(source.indexOf('await kernel.use(serverPlugin);')) + .toBeLessThan(source.indexOf('await kernel.use(devArtifactDoor);')); + // ...and the wrap is constructed BEFORE it, which is the whole reason + // the decision had to be hoisted. + expect(source.indexOf("devArtifactDoor ? { securityMetadataRegistrar: 'artifact-door' } : {},")) + .toBeLessThan(source.indexOf('await kernel.use(devArtifactDoor);')); + }); + + it('the `kernel.use` catch does not claim a consequence it cannot have', () => { + // An earlier draft warned there that the four collections had gone + // unregistered. `Kernel.use` only validates the plugin and registers + // it by NAME (packages/core/src/kernel.ts) — `init`/`start` run later, + // in `bootstrap` — so for a MetadataPlugin already constructed above, + // on a still-`idle` kernel, that catch does not fire. The real + // lost-door case is the missing artifact, and it warns where the path + // can be named; see the existence test above. + const useAt = source.indexOf('await kernel.use(devArtifactDoor);'); + expect(useAt).toBeGreaterThan(-1); + const catchWindow = source.slice(useAt, useAt + 600); + expect(catchWindow).not.toContain('The app wrap deferred'); + expect(catchWindow).not.toContain('NOT registered on this boot'); + }); + + it('behavioural: the option the source passes is the one AppPlugin reads', () => { + const bundle = { manifest: { id: 'com.test.14397', name: 'pin', version: '1.0.0' } }; + // The exact two literals the composition above can pass. + expect(new AppPlugin(bundle, undefined, {}).securityMetadataRegistrar).toBe('app-plugin'); + expect( + new AppPlugin(bundle, undefined, { securityMetadataRegistrar: 'artifact-door' }) + .securityMetadataRegistrar, + ).toBe('artifact-door'); + }); +}); diff --git a/packages/cli/src/commands/serve.ts b/packages/cli/src/commands/serve.ts index f618a92a82..86b5052c35 100644 --- a/packages/cli/src/commands/serve.ts +++ b/packages/cli/src/commands/serve.ts @@ -2794,6 +2794,106 @@ export default class Serve extends Command { const configHasMetadata = !!( config.objects || config.manifest || config.apps || config.flows || config.apis ); + + // ── Decide the dev-only artifact door BEFORE the wrap (#14397) ──── + // On a HOST config `os dev` composes TWO writers over ONE stack: the + // `new AppPlugin(config)` wrap below, over the config MODULE, and the + // dev-only HMR `MetadataPlugin` further down, over the compiled twin + // (`dist/objectstack.json`) the `os dev` supervisor produced from that + // same module. Measured on a real `os dev` boot of a host config: the + // door loads the artifact first and the wrap's ADR-0057 block registers + // last, so the WRAP's copy wins the cold boot — and the door's copy + // replaces it on the first artifact reload, so the winner CHANGES + // mid-run without a restart. ⚠️ The two copies differ by PROVENANCE and + // by FRESHNESS, not by parsing: on a config boot `defineStack()` is + // strict by default and runs the same `ObjectStackDefinitionSchema` + // parse the door runs, so the wrap's copy already carries the schema + // defaults and input transforms. What it lacks is the ADR-0010 stamp, + // and it never refreshes while the door's copy reloads on every + // recompile. Maintainer ruling (2026-08-29, #12892): the + // door owns the registration route; two permanent writers with matched + // shapes was refused as an end state. So the composition that runs the + // door declares it, here, exactly as `createStandaloneStack` does for + // the artifact boot. + // + // The door instance is CONSTRUCTED here rather than at its `kernel.use` + // site so that ONE value decides both facts — the wrap's registrar and + // the door's existence — instead of two expressions free to drift. + // Declaring the option with no door composed would be strictly worse + // than the divergence it removes: measured on `os serve` over the same + // host config, a metadata service is present and this wrap is its ONLY + // writer, so the four security collections would then have no registrar + // at all — the silent hole #12892 measured on the artifact boot. + // + // `hasMetadataPlugin` is read here instead of at the `kernel.use` site + // and the answer is identical: `plugins` is not mutated between the two + // points except by the AppPlugin append below, and an AppPlugin is not a + // MetadataPlugin. + let devArtifactDoor: any; + if (isDev && flags.server && !plugins.some((p: any) => p?.constructor?.name === 'MetadataPlugin')) { + try { + const { resolveDefaultArtifactPath } = await import('@objectstack/runtime'); + // `os dev` is the only caller that reaches here, and it is a + // supervisor: read its channel, or the artifact this HMR watcher + // polls would silently drift to `/dist/objectstack.json` + // whenever `os dev --artifact ` was used. + const hmrArtifactPath = resolveDefaultArtifactPath(readInternalArtifactPath()); + // ⛔ RESOLVING IS NOT EXISTING, and the difference decides who + // registers the four security collections. Under `os dev` the + // supervisor writes its channel unconditionally, so + // `readInternalArtifactPath()` always answers and + // `resolveDefaultArtifactPath` hands an explicitly named path back + // VERBATIM, with no existence check (packages/runtime/src/ + // default-host.ts — only the conventional `/dist/…` fallback is + // stat'ed). Compose a door over a path that is not there and it + // starts EMPTY and SILENT by design: `MetadataPlugin`'s local-file + // load is `{ optional: true }` and answers ENOENT with an `info` + // line, registering nothing (packages/metadata/src/plugin.ts). The + // wrap below would then have deferred to a door that never writes, + // and `positions` / `permissions` / `capabilities` / `sharingRules` + // would end the boot with ZERO registrars — green, quiet, and worse + // than the divergence this whole block exists to remove. So the + // door is composed only when its bytes are on disk NOW; a named but + // missing artifact leaves the wrap as the single registrar, exactly + // as before this decision existed. + // + // The trade this makes, stated rather than discovered later: on that + // boot the dev HMR SSE endpoint is not mounted either, because the + // plugin that mounts it is the one not composed. That is the right + // way round. The alternative — compose the door anyway and only + // withhold the registrar — keeps a dev convenience by re-creating the + // TWO-WRITER divergence this block exists to remove: the watcher + // picks the artifact up if it later appears, and then registers a + // second copy behind a wrap that already registered its own. The + // endpoint is missing only where `os dev` was pointed at an artifact + // that is not there (`--artifact` skips auto-compile), and the + // warning above says so by name. + if (hmrArtifactPath && !/^https?:\/\//i.test(hmrArtifactPath)) { + if (!fs.existsSync(hmrArtifactPath)) { + console.warn(chalk.yellow( + ` ⚠ Dev metadata-HMR endpoint not enabled: no compiled artifact at ${hmrArtifactPath}` + + '\n Stack-declared security metadata stays with the app wrap, its registrar on every' + + '\n boot without a door.', + )); + } else { + const { MetadataPlugin } = await import('@objectstack/metadata'); + // Mirror the standalone stack's dev config exactly + // (packages/runtime/src/standalone-stack.ts): declarative + // metadata is loaded from the compiled artifact — no source-file + // scanner (redundant + EMFILE-prone) — and `artifactWatch` polls + // the single artifact file so an `os dev` recompile broadcasts a + // reload over the SSE stream. + devArtifactDoor = new MetadataPlugin({ + watch: false, + artifactWatch: true, + artifactSource: { mode: 'local-file', path: hmrArtifactPath }, + }); + } + } + } catch (e: any) { + console.warn(chalk.yellow(` ⚠ Dev metadata-HMR endpoint not enabled: ${e?.message}`)); + } + } // ORDERING (#4085 → #4131/ADR-0116): the wrap is APPENDED to `plugins` // rather than registered here — but the append is no longer what makes // the boot correct. AppPlugin now DECLARES its ordering contract @@ -2808,7 +2908,15 @@ export default class Serve extends Command { if (!hasAppPluginAlready && configHasMetadata) { try { const { AppPlugin } = await import('@objectstack/runtime'); - plugins = [...plugins, new AppPlugin(config)]; + // The registrar is DECLARED by this composition, never inferred by + // AppPlugin — see the `devArtifactDoor` block above. Default + // (`'app-plugin'`) whenever no door composes, which is every + // non-dev boot of a host config. + plugins = [...plugins, new AppPlugin( + config, + undefined, + devArtifactDoor ? { securityMetadataRegistrar: 'artifact-door' } : {}, + )]; } catch (e: any) { // Non-fatal — the platform still boots, just without this app's // metadata. But it must SAY so: this catch was silent, and the two @@ -3011,36 +3119,27 @@ export default class Serve extends Command { // Registered AFTER the HonoServer plugin so the `http-server` service // (and its `getRawApp()`) is available when MetadataPlugin.start() mounts // the route. - if (isDev && flags.server) { - const hasMetadataPlugin = plugins.some( - (p: any) => p?.constructor?.name === 'MetadataPlugin' - ); - if (!hasMetadataPlugin) { - try { - const { resolveDefaultArtifactPath } = await import('@objectstack/runtime'); - // `os dev` is the only caller that reaches here, and it is a - // supervisor: read its channel, or the artifact this HMR watcher - // polls would silently drift to `/dist/objectstack.json` - // whenever `os dev --artifact ` was used. - const hmrArtifactPath = resolveDefaultArtifactPath(readInternalArtifactPath()); - if (hmrArtifactPath && !/^https?:\/\//i.test(hmrArtifactPath)) { - const { MetadataPlugin } = await import('@objectstack/metadata'); - // Mirror the standalone stack's dev config exactly - // (packages/runtime/src/standalone-stack.ts): declarative - // metadata is loaded from the compiled artifact — no source-file - // scanner (redundant + EMFILE-prone) — and `artifactWatch` polls - // the single artifact file so an `os dev` recompile broadcasts a - // reload over the SSE stream. - await kernel.use(new MetadataPlugin({ - watch: false, - artifactWatch: true, - artifactSource: { mode: 'local-file', path: hmrArtifactPath }, - })); - trackPlugin('Metadata'); - } - } catch (e: any) { - console.warn(chalk.yellow(` ⚠ Dev metadata-HMR endpoint not enabled: ${e?.message}`)); - } + // + // The instance was resolved and constructed next to the AppPlugin wrap + // (search `devArtifactDoor`) because the wrap's + // `securityMetadataRegistrar` has to be decided from the SAME value; + // only the `kernel.use` stays here, where the ordering requirement is. + // The catch is the pre-existing one and keeps its pre-existing message. + // ⛔ It is deliberately NOT the place to warn that the four security + // collections went unregistered: `Kernel.use` only validates the plugin + // and registers it by name (packages/core/src/kernel.ts) — `init` and + // `start` run later, in `bootstrap` — so for a MetadataPlugin that was + // constructed successfully a few hundred lines above, on a kernel still + // `idle`, this path does not fire. The registrar hand-off is decided by + // whether `devArtifactDoor` exists at all, and the case that genuinely + // loses a door — a named artifact that is not on disk — warns at the + // construction site, where the missing path can be named. + if (devArtifactDoor) { + try { + await kernel.use(devArtifactDoor); + trackPlugin('Metadata'); + } catch (e: any) { + console.warn(chalk.yellow(` ⚠ Dev metadata-HMR endpoint not enabled: ${e?.message}`)); } }