Skip to content

docs(audits): measure whether os create and os init should converge - #15797

Merged
os-litant merged 1 commit into
mainfrom
claude/issue-15531-measure-create-init-convergence
Sep 5, 2026
Merged

docs(audits): measure whether os create and os init should converge#15797
os-litant merged 1 commit into
mainfrom
claude/issue-15531-measure-create-init-convergence

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Part of #15531

The measurement #14824's ruling reserved. ⛔ This PR decides nothing and converges nothing — it adds one audit document and touches no emitted output, no published export and no command surface. The convergence decision stays the maintainer's; #15531 stays open for it.

Diff: one new file, docs/audits/2026-09-os-create-os-init-convergence-measurement.md. No package publishes from docs/audits/, so this carries the skip-changeset label rather than a changeset.

How the numbers were taken

Every file map is a real render, not a reading of the templates: a throwaway vitest harness drove Init.prototype.run() and Create.prototype.run() — the actual command bodies, parse() stubbed to supply argv — into throwaway directories outside the repository, and hashed the resulting trees. The harness was deleted before commit (git status clean); §10 of the document reproduces it so any reader can re-take the reading. The dependency closure was built first (pnpm --filter '@objectstack/lint...' build), because an unbuilt dist would have made the render a verdict about build state.

What the measurement found

1. The population is three scaffolders, not two. npx create-objectstack ships from its own published package and carries 13 live doc pages, against os create's 4 and os init's 3. ⚠️ The card's supporting claim that "os init is the documented on-ramp" is stale on this treecontent/docs/deployment/cli.mdx opens the first-app walkthrough with npm create objectstack@latest and carries an explicit "Which scaffolder?" callout casting os init as the alternative. The card's central claim is unaffected and confirmed.

2. The shared surface is exactly the four exports the card lists, all live, at six call sites, one-directional. Verified complete: no fifth export is imported and none has fallen out of use.

3. The drift history is confirmed with dates. create.ts carried workspace:* and the broken extends from 2026-01-31; init.ts learned the published-range pin on 2026-05-25; the restatement was corrected on 2026-09-04. ⇒ 102 days of drift on the values that decide whether a scaffold installs at all.

4. Of the four file names both commands emit, two already render byte-identical outputpnpm-workspace.yaml because it is imported (one hash across five emissions), and tsconfig.json by coincidence of two restatements that agree today.

5. ⭐ The word "plugin" names two different artifacts. os init -t plugin emits a metadata plugin (objectstack compile, private, declarative objects); os create plugin emits a code plugin (tsc, publishable, implements the kernel contract with init() / destroy()). Essential difference, and a live documentation defect independent of any convergence.

6. ⭐ There is exactly one genuine duplicate, and it is a template rather than a command family: os create example is a strictly weaker os init — same tsconfig.json bytes, same pnpm-workspace.yaml bytes, no install, no scaffold self-validation, no .gitignore, no name validation; everything it adds is one README.md.

7. The imported values have not drifted; the restated ones have. Three scaffolders declare three different TypeScript ranges today (^5.3.0 / ^5.8.0 / ^6.0.0), and two vitest ranges. Same defect class as the 102-day drift, one severity band lower.

8. The drift guards survive and simplify, exactly as the card predicted. Both both-scaffolder sweeps derive their populations from the two template maps rather than listing them, so a merge re-points them rather than deleting them. The create-objectstack consistency pin is untouched by any create/init merge — it holds a different pair.

Recommendation (input to the decision, not the decision)

Do not merge the command families. Leading on axis ② (long-term, ≥50% weight): the shape this repo is already converging on is one emission policy with thin front-ends that import it, and the measurement says the long-term risk is the restatement, not the command count. A merge spends a deprecation across 7 doc pages, a docs-parity pin and a CI path-filter parity script on the command count, and leaves the restatement where it is.

Instead, three separate follow-ups: extract the remaining restated policy (no user-visible change); ask the maintainer the one user-visible question that is actually open — should os create example be retired; and fix the plugin word collision in the docs. ⛔ This PR implements none of them.

Findings handed back rather than filed blind

Out of scope and untouched: #15530.

Verification

Gate union derived from the final commit with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack (9 families) and run at 7932cfc8fbc, exit codes captured after redirection, never through a pipe. All green, quoting each gate's own verdict line:

gate verdict
check:nul-bytes OK (scanned 7621 text file(s) ... no raw ASCII control bytes)
check:cross-package-test-inputs OK: 27 package(s) read outside themselves, all declared
check:doc-authoring ✓ doc authoring guard: sibling-package prose ids hold the baseline
check:merge-driver ✓ check-regen-pending self-test passed.
check-ci-filter-parity.mjs OK: all 149 declared cross-package glob(s) ... covered
check:doc-formula-expressions (lint) exit 0
check:empty-state (spec) ✓ all classified (2 closed, 2 open, 4 output, 8 scope)
check:liveness (spec) ✓ every governed-type property ... is classified
check:strictness-ledger (spec) ✓ strictness ledger: 60 file(s) across 5 triaged director(ies)
check:variant-docs (spec) ✓ variant/doc gate: 18 discriminated union(s)

No test was skipped, disabled or quarantined; PR #14821's both-scaffolder pin is untouched and stays the drift guard. No ablation was owed — no behaviour changed.


Generated by Claude Code

…ge (#15531)

The measurement #14824's ruling reserved. No emitted output, no published
export and no command surface changed — the convergence decision stays the
maintainer's.

Rendered both command bodies into throwaway trees and hashed the results:
of the four file names both emit, two already render byte-identical output;
the genuine duplicate is one template (`os create example`), not a command
family. Three scaffolders ship, not two — `npx create-objectstack` carries
13 doc pages to `os create`'s 4 and `os init`'s 3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation labels Sep 5, 2026
@os-litant os-litant added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed documentation Improvements or additions to documentation size/m labels Sep 5, 2026 — with Claude

Copy link
Copy Markdown
Collaborator Author

⚠️ Label write disclosed, because its read-back could not be taken.

skip-changeset was applied to this PR (the diff is one file under docs/audits/, which publishes from no package). Two caveats a reviewer should know rather than infer:

  • The additive write path is unavailable in this session — REST was probed before the first edit and answers 403 GitHub access is not enabled for this session, so the only available spelling is the whole-set update.
  • The read needed to compute a union first, and the read-back afterwards, both failed: API rate limit already exceeded on three separate attempts spanning the round. So this was a whole-set write of one label with no before-image and no after-image.

⇒ If a size:* (or any other bot-applied) label was present when this landed, it may have been displaced. Nothing gate-semantic should have been: this PR is minutes old and carries no other label this seat set. Please re-add anything missing rather than reading its absence as a decision — and the absence of skip-changeset itself, if the write did not stick, reads the same way.


Generated by Claude Code

@os-litant os-litant left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by the domain:cli execution seat (#6024). ⚠️ Recorded as a COMMENT rather than an approving review because GitHub refuses APPROVE on a PR authored under the same account — the dispatched dev runs as a subagent of this seat. Per the C4 SELF-REVIEW rule the branch (claude/issue-15531-…) is the dev's identity and the reviewing session is this seat's, so these cannot collide and the rule does not fire; the block above is GitHub's account check, not a governance verdict.

Landing pre-checks (2026-08-31 ruling), all three:

  • ① Clause ② graded no on the delivered diff: one added file under docs/audits/, no published package, no emitted output, no command surface. Neither the mechanical floor (no new key on a published payload, no packages/spec/src/**) nor the conformance limb applies.
  • needs:contract-review absent on both carriers — read back directly on the card (#15531: pm:dispatched, domain:cli, finding, priority:p3) and on this PR (documentation, skip-changeset).
  • All 45 checks green or skipped, not a required subset. Check Changeset went green at 10:05:05Z after the skip-changeset label landed.

⚠️ scripts/pm/check-clause2-carriers.mjs --pair could not be used: it exits 3 PREREQUISITE NOT MET in this container, and its own classifier (check-half-states.mjs --probe) reports repo-scoped reads refused at the egress proxy while /rate_limit returns 200 — the #9946 shape. Its stated fix is to keep repo-scoped reads on the MCP path, which is how both carriers above were read. Recorded so this is not mistaken for a machine-clean 0.

On the substance. The discipline that makes this document worth landing is that the file maps are a real render — the actual Init.prototype.run() / Create.prototype.run() bodies driven into throwaway directories and hashed — not a reading of the templates, with the dependency closure built first so the render is not a verdict about build state. The harness was deleted and is reproduced in §10, so the reading is re-takeable.

Three things earn their place:

  • It falsified its own card. §1 records that "os init is the documented on-ramp" is stale — create-objectstack carries 13 doc pages against 4 and 3 — and says so while leaving the card's central claim confirmed. A measurement that only confirms its brief is the one you cannot trust.
  • §5.3 narrows the question honestly. After §5.1 and §5.2, the measured duplicate is one template (os create example), not a command family. That is a smaller and more answerable question than the one the card posed.
  • §5.1plugin naming two different artifacts (metadata vs code plugin) is a live documentation defect found by rendering, invisible from the source.

⛔ Scope held exactly: this decides nothing and converges nothing. The convergence question stays the maintainer's and #15531 stays open for it; the three follow-ups (§9.1–3) are named and explicitly not implemented. The findings in §7 are handed back rather than filed blind.

Not a governed surface: docs/audits/** is outside docs/adr/**, .claude/**, skills/**, AGENTS.md, CLAUDE.md — and content/docs/releases/ is untouched. Flipped ready and auto-merge armed.


Generated by Claude Code

Merged via the queue into main with commit 1f1b38d Sep 5, 2026
49 of 50 checks passed
@os-litant
os-litant deleted the claude/issue-15531-measure-create-init-convergence branch September 5, 2026 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants