fix(types,runtime): log every 5xx at error level instead of answering it silently

[hotlong]

Fixes #14310

A 500 that leaves no server-side line is diagnosed from the browser or not at all. Reproduced on main @ ca48cf377, driving the real dispatcher plugin and the real route handlers: a plain Error thrown out of a route answered 500 INTERNAL_ERROR with zero log records at any level.

MEASURED status = 500
MEASURED logger.error calls = 0
MEASURED body = {"success":false,"error":{"code":"INTERNAL_ERROR","message":"boom-plain-error","httpStatus":500}}

The single logger.warn in that run was unrelated boot chatter (a fake transport with no setFallbackHandler), so the fault itself reached nobody at any level.

Why the existing machinery did not already cover this

Two independent reasons, both measured rather than reasoned:

1. ErrorReporter.captureException defaults to NoopErrorReporter. A dev server — the surface an operator actually watches — wires no APM, so the capture was a no-op every time. A log line is the operator's floor; APM is opt-in telemetry on top of it.
2. It is fed by res.__obsRecordedError, which only the THROWN exit sets. A route that catches its own fault and RETURNS a 5xx envelope — how every /packages handler answers, via deps.errorFromThrown — recorded nothing, so even a wired reporter never saw those. Confirmed in the same run: the thrown path had __obsRecordedError set = true, the returned path false.

Where it lands, and why that is the single conversion point

logServerFault (new, packages/types/src/server-fault-log.ts) emits exactly one error-level record carrying method, path, request id, the message and — where the door still holds the throw — the stack.

It shares a home with resolveThrownHttpError on that rule's own argument: a rule two doors must agree on cannot live inside one of them, because @objectstack/runtime depends on @objectstack/rest and an import could only ever point one way. That is why #8016 moved the "what status does this throw mean" rule to @objectstack/types; "is this answer worth an operator's attention" is the same kind of rule read by the same two doors.

Wired at each transport's own single exit, so a fault costs one line and never two:

Exit	File	Covers
sendError	packages/types/src/response-envelope.ts	every nested-envelope 5xx in the repo, incl. the REST direct-mount /api/v1/packages registrar
errorResponseBase	packages/runtime/src/dispatcher-plugin.ts	the dispatcher's THROWN exit (stack-bearing)
sendResultBase	packages/runtime/src/dispatcher-plugin.ts	the dispatcher's RETURNED exit — the /packages shape the card was filed on
AI-route mount	packages/runtime/src/dispatcher-plugin.ts	the family that writes its own result inline

sendError is the load-bearing choice for the REST side: it is the one writer every catch in the package registrar already ends in, so those doors become loud from the producer side with no per-door call — and a door added later cannot forget one. packages/rest's /data doors were already loud via logUnexpectedRouteError and are untouched.

Request coordinates reach the two dispatcher exits through a res.__obsRequest side-channel parked by instrumentRouteHandler — the same shape and lifetime as the __obsRecordedError channel that exit has used since #3867. That avoids threading a third argument through the ~50 route handlers that call the locally-shadowed sendResult / errorResponse wrappers.

Evidence that 4xx is unaffected

The 5xx test lives once, inside the helper, rather than at each call site — so no call site can drift from it.

• Unit: isServerFault pinned at the 499/500 boundary; logServerFault returns false and emits on no channel for a 404.
• Funnel: sendError at 409 DESTRUCTIVE_CHANGE and 403 FORBIDDEN — the coded refusals that door exists to carry — emit zero lines.
• End-to-end: an anonymous caller on an auth-gated dispatcher route (401) produces zero fault lines, measured through the real plugin.
• The wire body is byte-identical at every door; this adds a side effect, never a field (pinned).

error level is load-bearing rather than incidental: the CLI default is warn (DEFAULT_LOG_LEVEL, packages/cli/src/utils/log-level.ts) and error (40) outranks warn (30) in LEVEL_PRIORITY, so the record clears --log-level's default without bypassing the level system. The pins assert the level, not merely that output happened.

Reverse verification

Both landings were ablated from a committed tree, each mutation confirmed on disk by anchor count and object hash before the run, and restored by git checkout HEAD -- ABSOLUTE_PATH verified by empty git diff HEAD plus a worktree/HEAD blob-hash match:

• removing the call from errorResponseBase → the two thrown-path pins fail (expected [] to have a length of 1), the returned-path and 4xx pins stay green;
• removing the call from sendError → only the funnel pin fails (1 failed / 12 passed).

Each exit's pin is falsifiable by that exit alone.

Serial fence

Re-derived against the domain seat's fence on the card. packages/rest/src/package-routes.ts is held by the open PR #14499, so this PR does not touch it — the REST doors are covered from the producer side instead. Verified byte-identical to origin/main at this head: package-routes.ts, error-response.ts (PR #14544, merged), dispatcher-error-vocabulary.ts (PR #14625, open).

The #7898 H17 rider does not fire: this diff touches none of packages/core/src/security/auth-gate.ts, packages/runtime/src/http-dispatcher.ts, or packages/adapters/**.

Verification (all at b4bc2ab47)

• pnpm --filter @objectstack/types exec vitest run — 19 files, 548 passed
• runtime sweep (dispatcher-plugin*, dispatcher-5xx*, instrument, validation-error, withhold, package-door parity, error-envelope conformance) — 21 files, 285 passed
• typecheck on types + runtime + rest — clean
• Gate union re-derived with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (no paths passed) — 35 families, 31 exit 0. The other four exit 3 = PREREQUISITE NOT MET, not findings: check-test-completeness, check-half-states, check:dual-build-cjs-loads, check:type-check-debt each need a full-repo build or GitHub API access and print "nothing was measured". CI runs them.

⚠️ NOT MEASURED, stated rather than implied: packages/runtime/tsconfig.json excludes **/*.test.ts, so pnpm --filter @objectstack/runtime typecheck says nothing about the new test file (tsc --listFiles → 0 matches). Both new test files were type-checked explicitly under a temporary config (0 errors) and the temp configs removed.

Declared narrowing — pnpm lint (whole-repo ESLint) was not run. Targeted instead: eslint --no-inline-config --format json over all 7 changed .ts files — 0 errors, 0 warnings. The three facts that make this a measurement rather than a gap: (a) the file count is read from ESLint's own --format json output, not estimated; (b) the population is read from the repo's config, which never enables type-aware linting for any file (no parserOptions.project, no typed @typescript-eslint rules — eslint.config.mjs, with its own recorded positive control); (c) therefore no untouched file's verdict can move as a result of this diff.

Declared narrowing — verification ran UNLOCKED. scripts/pm/os-verify-lock.sh
could not take the shared verify lock on this host: no usable flock. The shared
verify lock is declared Linux-only (flock is util-linux, and a stock macOS does
not ship it), so the command below was run directly, without the lock —
a declared narrowing, not a silent one. No serialization guarantee held for this
run, nor for any sibling agent in this container while it ran.

One consequence worth a maintainer's eye

A deployment answering a declared 5xx on a polled route — 501 NOT_IMPLEMENTED from an uninstalled optional service, e.g. /api/v1/notifications without service-messaging — now prints one error line per request where it previously printed none. This PR implements the band the issue specifies verbatim ("4xx may stay quiet; 5xx never") and deliberately invents no carve-out: narrowing it for declared capability-absence would be a contract decision, not an implementer's call.

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/runtime, @objectstack/types, touching 22 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/types/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

15 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

• content/docs/api/client-sdk.mdx (via requestId (symbol, a field of interface ServerFaultRequest), packages.list (sdk, the route ledger binds it to GET /api/v1/packages))
• content/docs/api/environment-routing.mdx (via packages.list (sdk, the route ledger binds it to GET /api/v1/packages))
• content/docs/api/error-catalog.mdx (via requestId (symbol, a field of interface ServerFaultRequest))
• content/docs/api/error-handling-client.mdx (via requestId (symbol, a field of interface ServerFaultRequest))
• content/docs/automation/approvals.mdx (via requestId (symbol, a field of interface ServerFaultRequest))
• content/docs/deployment/production-readiness.mdx (via createDispatcherPlugin (symbol, a top-level function), requestId (symbol, a field of interface ServerFaultRequest))
• content/docs/getting-started/examples.mdx (via /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))
• content/docs/kernel/contracts/metadata-service.mdx (via /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))
• content/docs/kernel/runtime-services/audit-service.mdx (via requestId (symbol, a field of interface ServerFaultRequest))
• content/docs/kernel/services-checklist.mdx (via /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))
• content/docs/permissions/permission-sets.mdx (via /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))
• content/docs/plugins/packages.mdx (via createDispatcherPlugin (symbol, a top-level function))
• content/docs/protocol/kernel/error-handling.mdx (via requestId (symbol, a field of interface ServerFaultRequest))
• content/docs/protocol/kernel/http-protocol.mdx (via /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))
• content/docs/ui/apps.mdx (via /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))

⛔ 1 release-owned page(s) also name something this change touched. These are read-only:

• content/docs/releases/v17.mdx (via requestId (symbol, a field of interface ServerFaultRequest), /api/v1/packages (route, a path literal in ServerFaultRequest; a path literal in sendError; a path literal on a changed line))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see

• 1 changed file(s) yielded no anchor (packages/types/src/index.ts) — pages documenting those are invisible to this run
• 2 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 47 of 219 client-bound route-ledger rows — the other 172 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 172: 14 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 24 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 20b883918aa787e65299e72b85c2b3396aa2006f → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 25beb23d6384a4a100200ed6882330e0b71ff003 — the merge of head b4bc2ab473e5f8b9d95326e9a191f4ded2ac98e5 into base 20b883918aa787e65299e72b85c2b3396aa2006f, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 25beb23d6384a4a100200ed6882330e0b71ff003 && git checkout 25beb23d6384a4a100200ed6882330e0b71ff003
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 20b883918aa787e65299e72b85c2b3396aa2006f b4bc2ab473e5f8b9d95326e9a191f4ded2ac98e5 && git checkout -B drift-repro 20b883918aa787e65299e72b85c2b3396aa2006f && git merge --no-ff b4bc2ab473e5f8b9d95326e9a191f4ded2ac98e5

node scripts/docs-audit/affected-docs.mjs --json 20b883918aa787e65299e72b85c2b3396aa2006f

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 20b883918aa787e65299e72b85c2b3396aa2006f → pass the list as
args.docs, on the commit named under Which tree this was computed on.