fix(objectql,runtime,rest): store a serializable manifest projection in the package registry

.changeset/packages-registry-serializable-manifest.md

@@ -0,0 +1,51 @@
+---
+"@objectstack/objectql": patch
+"@objectstack/runtime": patch
+"@objectstack/rest": patch
+---
+
+fix(objectql,runtime,rest): store a serializable manifest in the package registry so `/packages` stops answering 500 (#14309)
+
+On a stock showcase boot, signed in as the seeded admin, every read door that
+serialises a package answered `500 INTERNAL_ERROR`:
+
+```
+GET /api/v1/packages                           -> 500
+GET /api/v1/packages/com.example.showcase      -> 500
+GET /api/v1/meta/package/com.example.showcase  -> 500
+GET /api/v1/meta/package/com.objectstack.setup -> 200
+```
+
+with `Converting circular structure to JSON · _ObjectQL -> actionActivation ->
+store -> engine`. Studio asks for the list three times on every open.
+
+**Cause.** `SchemaRegistry.installPackage(manifest)` kept the caller's object
+verbatim as `pkg.manifest`. For a code-defined stack that object is the live
+`defineStack()` one, and its `plugins: [new ConnectorRestPlugin(), …]` entries
+hold the engine once they initialise — a cycle since the engine grew
+`actionActivation -> store -> engine`. Measured on that boot: of the 26
+installed packages exactly ONE manifest key was unserializable (`plugins`, on
+`com.example.showcase`), and only after plugin init — during boot the same
+manifest serialised cleanly, which is why a package with no plugin instances
+(`com.objectstack.setup`) kept answering 200.
+
+**Fix, at the producer.** `installPackage` now stores a serializable projection:
+the registry item is a record, not the runtime. The projection drops by shape
+rather than by key name — functions, class instances, `Map`/`Set` and reference
+cycles are dropped; primitives, plain objects, arrays and `Date` survive — so a
+future live member cannot re-open the same hole. The kernel keeps the live
+object (`ObjectQL.manifests`), and the one reader of `manifest.plugins[]` reads
+its own parameter, never the record, so nothing downstream loses a member it was
+using. The caller's manifest is copied, never stripped in place.
+
+**Defence at the read doors.** `GET /packages` and `GET /packages/:id` project a
+registry entry onto its declared record fields instead of spreading it whole, so
+an undeclared member appearing on the *item* degrades to a field the response
+never mentions instead of failing the whole list for every caller. Applied at
+both twins — `packages/runtime/src/domains/packages.ts` (the handler that
+actually answered the 500; the 404 wording identifies it) and the
+`packages/rest` routes. The database half of the REST merge is deliberately not
+projected: its shape belongs to `PackageService`.
+
+No response field is added or renamed. Responses that already served fine are
+byte-identical; what disappears is a member that could never be serialised.

content/docs/permissions/system-context.mdx

@@ -135,7 +135,7 @@ The largest single consumer — **20 of the 109 sites**.
 | 34 | `revoke()` deletes directly, **before** the non-manual-source guard | Get: the evaluator can revoke its own grants. Lose: the `CONFLICT` guard that warns a rule-materialised grant will be silently re-granted on the next reconcile | `plugin-sharing/src/sharing-service.ts:1286` (guard at `:1311`) |
 | 35 | `listShares()` skips the management gate | Get: full enumeration of who can see a record | `plugin-sharing/src/sharing-service.ts:1338` |
 | 36 | `sys_record_share` reads are **not** self-scoped | Get: tenant-wide share listing without `manage_sharing` | `sharing-plugin.ts:1077` |
-| 37 | Share-link policy `enabled` check bypassed; system callers re-enter under a system context | Get: link **creation** while the policy is off — resolution is **not** bypassed since #14033 (`publicSharing.enabled` is a standing policy held at every redemption): a link minted this way does not resolve until the block is enabled | `plugin-sharing/src/share-link-service.ts:449`, `:503`, `:507`, `:580`, `:610` |
+| 37 | Share-link policy `enabled` check bypassed; system callers re-enter under a system context | Get: link creation/resolution while the policy is off | `plugin-sharing/src/share-link-service.ts:449`, `:503`, `:507`, `:580`, `:610` |
 | 38 | Sharing-rule provenance stamp skipped | Lose: the row is not marked as an admin customization — seeder / `defineRule` / boot reconcilers are "the package door" | `sharing-rule-provenance.ts:47` |
 | 39 | Sharing-rule service write + delete paths return early | Lose: the manage-rules gate on the service surface, and the platform-global-rule delete guard | `sharing-rule-service.ts:157`, `:382` |
 
@@ -160,10 +160,10 @@ The largest single consumer — **20 of the 109 sites**.
 | 49 | Action `requiredPermissions` bypassed | runtime | Get: engine self-invocation runs any action | `action-execution.ts:399` |
 | 50 | `manage_metadata` bypassed on metadata writes | runtime, rest | Get: schema writes without the capability | `domains/meta.ts:471`, `:874`, `rest-server.ts:4716`, `:6079`, `:6327`, `:6758`, `:6951` |
 | 51 | The shared metadata-write verdict itself returns `allowed` | metadata-core | Get: the one function all of row 50's doors consult answers yes before any capability is examined | `meta-write-capability.ts:134` |
-| 52 | Anonymous-deny seam satisfied on the domain dispatchers and the package/federation routes | runtime, rest | Get: passes with no `userId` | `domains/actions.ts:411`, `domains/ai.ts:60`, `domains/automation.ts:989`, `domains/meta.ts:232`, `domains/security.ts:78`, `domains/packages.ts:276`, `external-datasource-routes.ts:302`, `package-routes.ts:97` |
+| 52 | Anonymous-deny seam satisfied on the domain dispatchers and the package/federation routes | runtime, rest | Get: passes with no `userId` | `domains/actions.ts:411`, `domains/ai.ts:60`, `domains/automation.ts:989`, `domains/meta.ts:232`, `domains/security.ts:78`, `domains/packages.ts:326`, `external-datasource-routes.ts:302`, `package-routes.ts:97` |
 | 53 | MCP principal check satisfied | runtime | Get: MCP surface reachable with no user | `domains/mcp.ts:61` |
 | 54 | Package REST route capability gate bypassed | rest | Get: package read/write over REST without `manage_metadata` / `studio.access` / `setup.access` | `package-routes.ts:102` |
-| 55 | Package domain capability gates bypassed | runtime | Get: package management and package-inventory reads without the capability | `domains/packages.ts:95`, `:128` |
+| 55 | Package domain capability gates bypassed | runtime | Get: package management and package-inventory reads without the capability | `domains/packages.ts:145`, `:178` |
 | 56 | Activation write / authoring refusals do not fire | runtime | Get: activation artifacts writable and authorable without the activation-authoring capability | `activation-gate.ts:139`, `:190` |
 | 57 | Automation run-state read, flow-authoring write and unrelated-screen read all pass | runtime | Get: run state, flow writes and screen reads with no grant | `domains/automation.ts:254`, `:545`, `:635` |
 | 58 | Audience-binding suggestion recording skipped | plugin-security | Lose: install-time suggestions are not recorded for system callers | `suggested-audience-bindings.ts:703` |

packages/objectql/src/registry-package-manifest-serializable.test.ts

@@ -0,0 +1,194 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * `installPackage` stores a RECORD, never the caller's live object.
+ *
+ * ## What was wrong
+ *
+ * `installPackage(manifest)` kept the argument verbatim as `pkg.manifest`. For a
+ * code-defined stack that argument is the live `defineStack()` object, and its
+ * `plugins: [new ConnectorRestPlugin(), …]` entries hold the engine once they
+ * initialise. Since the engine grew `actionActivation -> store -> engine` that
+ * reference closes a CYCLE, so `JSON.stringify` of the registry item threw and
+ * every read door that serialises a package answered `500 INTERNAL_ERROR` on a
+ * stock showcase boot — `GET /packages`, `GET /packages/:id`,
+ * `GET /meta/package/:id`, while `GET /meta/package/<a plugin-less package>`
+ * stayed 200.
+ *
+ * ## Why the assertions are shaped this way
+ *
+ * ⚠️ "the manifest still round-trips" passes on the old code for every package
+ * that has no plugins, which is 25 of the 26 a showcase boot installs. So the
+ * cases below pin the MECHANISM: a live instance reaching the record, a plain
+ * reference cycle, and a member that is not data at all — each asserted on
+ * `JSON.stringify(registry.getPackage(id))`, the exact expression the doors run.
+ *
+ * ⚠️ Timing is part of the defect and is pinned too. Measured on the failing
+ * boot: the same showcase manifest serialised CLEANLY during boot and only
+ * became cyclic after plugin init, so a check that ran at install time would
+ * have called the record healthy. The `becomes cyclic only after init` case
+ * below reproduces that ordering — the projection must not depend on when it is
+ * asked.
+ */
+
+import { describe, it, expect, beforeEach } from 'vitest';
+import { SchemaRegistry } from './registry';
+
+/**
+ * A plugin instance in the shape that broke: a class instance the host
+ * constructs in `objectstack.config.ts` and hands to `defineStack({ plugins })`,
+ * which takes the engine when it initialises.
+ */
+class FakeConnectorPlugin {
+  name = 'connector-rest';
+  engine: unknown;
+  init(engine: unknown) {
+    this.engine = engine;
+  }
+}
+
+/** The engine's own `actionActivation -> store -> engine` cycle, reproduced. */
+function makeCyclicEngine(): Record<string, unknown> {
+  const engine: Record<string, unknown> = { name: '_ObjectQL' };
+  const store: Record<string, unknown> = { name: 'ObjectStoreActionActivationStore', engine };
+  engine.actionActivation = { name: 'ActionActivationProjection', store };
+  return engine;
+}
+
+function baseManifest(overrides: Record<string, unknown> = {}): any {
+  return {
+    id: 'com.example.showcase',
+    name: 'Showcase',
+    namespace: 'showcase',
+    version: '1.2.3',
+    type: 'app',
+    scope: 'user',
+    description: 'Kitchen-sink example',
+    dependencies: ['com.objectstack.plugin-auth'],
+    objects: [{ name: 'invoice', fields: { total: { type: 'currency' } } }],
+    apps: [{ name: 'showcase', label: 'Showcase' }],
+    ...overrides,
+  };
+}
+
+describe('SchemaRegistry.installPackage — the record is serializable', () => {
+  let registry: SchemaRegistry;
+
+  beforeEach(() => {
+    registry = new SchemaRegistry({ multiTenant: false, collisionPolicy: 'error' });
+    registry.logLevel = 'silent';
+  });
+
+  it('survives a plugin instance that closes a cycle through the engine', () => {
+    const plugin = new FakeConnectorPlugin();
+    plugin.init(makeCyclicEngine());
+    registry.installPackage(baseManifest({ plugins: [plugin] }));
+
+    // The expression every read door runs.
+    expect(() => JSON.stringify(registry.getPackage('com.example.showcase'))).not.toThrow();
+    // The instance itself is gone from the record — this is a projection, not a
+    // replacement value that still points at the engine.
+    expect(JSON.stringify(registry.getPackage('com.example.showcase'))).not.toContain('_ObjectQL');
+  });
+
+  it('survives a manifest that becomes cyclic only AFTER install', () => {
+    // The measured ordering: at install the plugin holds no engine and the
+    // manifest serialises fine; the cycle appears when the plugin initialises.
+    // A projection taken at install must still hold, because it copied out of
+    // the live object rather than aliasing it.
+    const plugin = new FakeConnectorPlugin();
+    const manifest = baseManifest({ plugins: [plugin] });
+    expect(() => JSON.stringify(manifest)).not.toThrow();
+
+    registry.installPackage(manifest);
+    plugin.init(makeCyclicEngine());
+
+    expect(() => JSON.stringify(registry.getPackage('com.example.showcase'))).not.toThrow();
+  });
+
+  it('survives a reference cycle among PLAIN data in the manifest', () => {
+    const cyclic: Record<string, unknown> = { name: 'self' };
+    cyclic.self = cyclic;
+    registry.installPackage(baseManifest({ data: { node: cyclic } }));
+
+    const record = registry.getPackage('com.example.showcase')!;
+    expect(() => JSON.stringify(record)).not.toThrow();
+    // The back-edge is dropped; everything ahead of it survives.
+    expect((record.manifest as any).data.node.name).toBe('self');
+    expect((record.manifest as any).data.node.self).toBeUndefined();
+  });
+
+  it('keeps the declarative half of the manifest byte-for-byte', () => {
+    const manifest = baseManifest({ plugins: [new FakeConnectorPlugin()] });
+    registry.installPackage(manifest);
+    const stored = registry.getPackage('com.example.showcase')!.manifest as any;
+
+    for (const key of [
+      'id', 'name', 'namespace', 'version', 'type', 'scope', 'description',
+      'dependencies', 'objects', 'apps',
+    ]) {
+      expect(stored[key]).toEqual((manifest as any)[key]);
+    }
+  });
+
+  it('drops members that are not data, and keeps a Date as data', () => {
+    const publishedAt = new Date('2026-09-02T00:00:00.000Z');
+    registry.installPackage(baseManifest({
+      onEnable: () => undefined,
+      registryHandle: new Map([['a', 1]]),
+      publishedAt,
+    }));
+    const stored = registry.getPackage('com.example.showcase')!.manifest as any;
+
+    expect(stored.onEnable).toBeUndefined();
+    expect(stored.registryHandle).toBeUndefined();
+    expect(new Date(stored.publishedAt).toISOString()).toBe(publishedAt.toISOString());
+  });
+
+  it('does not mutate the caller’s manifest — the kernel keeps the live object', () => {
+    // `ObjectQL.registerApp` reads `manifest.plugins[]` from ITS OWN parameter
+    // to register nested plugins, and hands the same object to `installPackage`.
+    // Projecting must therefore copy, never strip in place.
+    const plugin = new FakeConnectorPlugin();
+    const manifest = baseManifest({ plugins: [plugin] });
+    registry.installPackage(manifest);
+
+    expect(manifest.plugins).toHaveLength(1);
+    expect(manifest.plugins[0]).toBe(plugin);
+    expect(registry.getPackage('com.example.showcase')!.manifest).not.toBe(manifest);
+  });
+
+  it('projects on REINSTALL too (rebuild / HMR overwrite)', () => {
+    registry.installPackage(baseManifest());
+    const plugin = new FakeConnectorPlugin();
+    plugin.init(makeCyclicEngine());
+    registry.installPackage(baseManifest({ plugins: [plugin] }));
+
+    expect(() => JSON.stringify(registry.getPackage('com.example.showcase'))).not.toThrow();
+  });
+
+  it('keeps the whole LIST serializable when one package carries the cycle', () => {
+    // The list door's failure mode: one unserializable item took out every
+    // caller's whole listing, not just the offending package.
+    const plugin = new FakeConnectorPlugin();
+    plugin.init(makeCyclicEngine());
+    registry.installPackage(baseManifest({ plugins: [plugin] }));
+    registry.installPackage({
+      id: 'com.objectstack.setup', name: 'Setup', namespace: 'setup', version: '9.3.0',
+    } as any);
+
+    expect(() => JSON.stringify(registry.getAllPackages())).not.toThrow();
+    expect(registry.getAllPackages()).toHaveLength(2);
+  });
+
+  it('still records the namespace and the lifecycle state it always did', () => {
+    // The projection must not cost the record its non-manifest half.
+    registry.installPackage(baseManifest({ plugins: [new FakeConnectorPlugin()] }));
+    const record = registry.getPackage('com.example.showcase')!;
+
+    expect(record.status).toBe('installed');
+    expect(record.enabled).toBe(true);
+    expect(typeof record.installedAt).toBe('string');
+    expect(registry.getNamespaceOwners('showcase')).toEqual(['com.example.showcase']);
+  });
+});