Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
195 changes: 156 additions & 39 deletions packages/cli/test/init-template-comments-self-contained.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
//
// Every comment that ships into a project scaffolded by `objectstack init`
// must be followable by the person reading it — someone who has that
// project and nothing else.
// Every comment that ships into a project this package scaffolds must be
// followable by the person reading it — someone who has that project and
// nothing else. This package has TWO scaffolders and both ship such text,
// so both are swept: `objectstack init` and `objectstack create`.
//
// ## The defect
//
Expand All @@ -16,19 +17,47 @@
// bundled template *files*; this is the OTHER scaffolder, which renders its
// templates as in-source string literals instead.
//
// ## Why the population spans BOTH scaffolders
//
// `packages/cli/src/commands/create.ts` exports a second, independent
// template map (`templates`) whose entries render text — an
// `objectstack.config.ts`, a plugin `src/index.ts`, a `README.md` — as
// in-source string literals and write them straight into the user's
// project. Same emitter shape, same population: text a scaffolded project
// actually receives. While this pin read only `init`'s map, its three
// assertions held for `init` only, and an edit to a `create` literal that
// cited an ADR or linked a docs page that later moved shipped to every
// `os create` user with every gate green. So the population is DERIVED
// from both maps — never written down — the way
// `scaffold-manifest-schema.test.ts` derives its own sweep: a template
// added to either map is swept the day it is added, with nobody
// remembering to extend this file.
//
// `create` has no `configContent` / `srcFiles` split. Every file it writes
// lives in one `files` map keyed by the path it lands at and is rendered by
// calling that entry, so this pin renders EVERY entry and serialises each
// one exactly the way `Create.run()` does (a string verbatim, anything else
// through `JSON.stringify(_, null, 2)`). Sweeping the whole map rather than
// a chosen subset is deliberate: a filter is a place a future file can
// escape through silently, which is the shape of the defect above.
//
// ## Why the population is the RENDERED output, not the source file
//
// `init.ts` also carries its own ordinary source comments that legitimately
// cite ADRs and issue numbers (e.g. the `printCreatedFilesSummary` doc
// comment cites #10499) — those never ship, because they live outside the
// `configContent` / `srcFiles` functions the command actually writes to
// disk. A pin that greps `init.ts` wholesale would match those too and
// report on the wrong population. So this pin does not read the source
// file at all: it calls the exact functions the `init` command calls
// (`template.configContent(...)`, `writeTemplateSrcFiles(...)`) and scans
// the files they actually write — the same real emitter
// `init-scaffold-authoring-rules.test.ts` uses, for the same reason (so
// neither test can drift from what `init` really does).
// disk. `create.ts` is the same: its `run()` body cites `packages/plugins`
// as a destination directory, which ships nowhere. A pin that grepped
// either source file wholesale would match those too and report on the
// wrong population. So this pin does not read the source files at all: it
// calls the exact functions each command calls
// (`template.configContent(...)`, `writeTemplateSrcFiles(...)`, and for
// `create` the entries of `template.files`) and scans the files they
// actually write — the same real emitters
// `init-scaffold-authoring-rules.test.ts` and
// `scaffold-manifest-schema.test.ts` use, for the same reason (so no test
// can drift from what the commands really do).
//
// ## Why this pin has TWO halves, and why the second is the load-bearing one
//
Expand Down Expand Up @@ -58,6 +87,7 @@ import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { TEMPLATES, sanitizeNamespace, writeTemplateSrcFiles } from '../src/commands/init.js';
import { templates as createTemplates } from '../src/commands/create.js';

const HERE = path.dirname(fileURLToPath(import.meta.url));
const TMP_ROOT = path.resolve(HERE, '../tmp');
Expand All @@ -70,43 +100,99 @@ afterAll(() => {
});

interface Rendered {
templateKey: string;
/** `<command>:<template key>` — the command a user would have typed. */
scaffoldId: string;
file: string;
content: string;
/**
* The template entry rendered a STRING — an in-source literal, the only
* kind of emitted file that can carry prose. Used by the vacuity guard
* so "this arm was swept" cannot be satisfied by serialised JSON alone.
*/
fromLiteral: boolean;
}

/** A throwaway project directory, registered for cleanup. */
function makeRoot(scaffoldId: string): string {
fs.mkdirSync(TMP_ROOT, { recursive: true });
const root = fs.mkdtempSync(path.join(TMP_ROOT, `render-${scaffoldId.replace(':', '-')}-`));
roots.push(root);
return root;
}

/**
* Render every built-in template through `init`'s own emitter — the exact
* functions the command calls, writing to real files in a throwaway
* directory (mirroring `init-scaffold-authoring-rules.test.ts`) — and
* return every file it produced. This IS the population the defect lives
* in: text a scaffolded project actually receives.
* Render every built-in template of BOTH scaffolders through its own
* emitter — the exact functions each command calls, writing to real files
* in a throwaway directory (mirroring `init-scaffold-authoring-rules.test.ts`)
* — and return every file they produced. This IS the population the defect
* lives in: text a scaffolded project actually receives.
*/
function renderAll(): Rendered[] {
const namespace = sanitizeNamespace(PROJECT_NAME);
const out: Rendered[] = [];
fs.mkdirSync(TMP_ROOT, { recursive: true });

const collect = (scaffoldId: string, root: string, literals: Set<string>) => {
const walk = (dir: string) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const abs = path.join(dir, entry.name);
if (entry.isDirectory()) walk(abs);
else {
const file = path.relative(root, abs);
out.push({
scaffoldId,
file,
content: fs.readFileSync(abs, 'utf8'),
fromLiteral: literals.has(file),
});
}
}
};
walk(root);
};

// ── `os init -t <key>`: every template renders a config plus src files ──
for (const templateKey of Object.keys(TEMPLATES)) {
const template = TEMPLATES[templateKey];
const root = fs.mkdtempSync(path.join(TMP_ROOT, `render-${templateKey}-`));
roots.push(root);
const root = makeRoot(`init:${templateKey}`);

fs.writeFileSync(
path.join(root, 'objectstack.config.ts'),
template.configContent(PROJECT_NAME, namespace),
);
writeTemplateSrcFiles(template.srcFiles, root, PROJECT_NAME, namespace);

const walk = (dir: string) => {
// Everything `init` emits here is an in-source string literal.
const literals = new Set<string>();
const markAll = (dir: string) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const abs = path.join(dir, entry.name);
if (entry.isDirectory()) walk(abs);
else out.push({ templateKey, file: path.relative(root, abs), content: fs.readFileSync(abs, 'utf8') });
if (entry.isDirectory()) markAll(abs);
else literals.add(path.relative(root, abs));
}
};
walk(root);
markAll(root);
collect(`init:${templateKey}`, root, literals);
}

// ── `os create <key> <name>`: one `files` map keyed by destination path ─
for (const templateKey of Object.keys(createTemplates)) {
const template = createTemplates[templateKey as keyof typeof createTemplates];
const root = makeRoot(`create:${templateKey}`);
const files = template.files as Record<string, (name: string) => unknown>;
const literals = new Set<string>();

for (const [filePath, render] of Object.entries(files)) {
const content = render(PROJECT_NAME);
// Exactly what `Create.run()` writes for this entry.
const text = typeof content === 'string' ? content : JSON.stringify(content, null, 2);
if (typeof content === 'string') literals.add(filePath);
const abs = path.join(root, filePath);
fs.mkdirSync(path.dirname(abs), { recursive: true });
fs.writeFileSync(abs, text);
}
collect(`create:${templateKey}`, root, literals);
}

return out;
}

Expand All @@ -124,17 +210,36 @@ const MONOREPO_ONLY = [
{ label: 'a monorepo package path', re: /\bpackages\/[a-z0-9][\w-]*\//i },
];

describe('rendered init templates are followable by a stranger', () => {
describe('rendered scaffold templates are followable by a stranger', () => {
const rendered = renderAll();

// ── vacuity guard: prove this is reading real rendered output ──────────
it('rendered a real, non-empty project per template (vacuity guard)', () => {
it('rendered a real, non-empty project per template of BOTH scaffolders (vacuity guard)', () => {
expect(Object.keys(TEMPLATES).length).toBeGreaterThan(0);
expect(Object.keys(createTemplates).length).toBeGreaterThan(0);
expect(rendered.length).toBeGreaterThan(0);

for (const templateKey of Object.keys(TEMPLATES)) {
const files = rendered.filter((r) => r.templateKey === templateKey);
expect(files.map((f) => f.file), `template "${templateKey}"`).toContain('objectstack.config.ts');
const files = rendered.filter((r) => r.scaffoldId === `init:${templateKey}`);
expect(files.map((f) => f.file), `template "init:${templateKey}"`).toContain('objectstack.config.ts');
}

// `create`'s templates are NOT checked for that one filename: its
// `plugin` template emits no `objectstack.config.ts` at all (its
// `src/index.ts` declares a `Plugin` object instead), so requiring one
// would report on a surface that scaffolder does not have. What must
// hold for every `create` template is that the sweep reached its
// in-source LITERALS — the only emitted files that can carry prose —
// which is where this pin's three assertions have anything to read.
for (const templateKey of Object.keys(createTemplates)) {
const literals = rendered.filter((r) => r.scaffoldId === `create:${templateKey}` && r.fromLiteral);
expect(
literals.length,
`template "create:${templateKey}" contributed no rendered string literal — the sweep ` +
'reached none of its prose, so every assertion below passes vacuously for it',
).toBeGreaterThan(0);
}

// The two templates that emit an object (app, plugin) must have reached
// the OWD comment's file, or assertion 2 below would vacuously pass.
// Selected STRUCTURALLY (anything under src/objects/ that is not the
Expand All @@ -145,16 +250,28 @@ describe('rendered init templates are followable by a stranger', () => {
expect(objectFiles.length).toBeGreaterThan(0);
});

// The second scaffolder, named — so a future edit that drops it from the
// population fails with this card's own vocabulary rather than a bare
// count that a shrinking sweep satisfies just as well.
it('sweeps the `os create` scaffolder, not just `os init`', () => {
const ids = [...new Set(rendered.map((r) => r.scaffoldId))];
expect(ids).toContain('create:example');
expect(ids).toContain('create:plugin');
expect(ids.filter((id) => id.startsWith('init:')).length).toBe(Object.keys(TEMPLATES).length);
expect(ids.filter((id) => id.startsWith('create:')).length).toBe(Object.keys(createTemplates).length);
});

// ── assertion 1: nothing unfollowable ───────────────────────────────────
it.each(rendered.map((r) => [`${r.templateKey}/${r.file}`, r] as const))(
it.each(rendered.map((r) => [`${r.scaffoldId}/${r.file}`, r] as const))(
'%s cites nothing that only exists in this monorepo',
(_label, r) => {
const command = r.scaffoldId.startsWith('init:') ? 'os init' : 'os create';
for (const { label, re } of MONOREPO_ONLY) {
const hit = re.exec(r.content);
expect(
hit,
`${r.templateKey}/${r.file} cites ${label} (${JSON.stringify(hit?.[0])}). A project ` +
'scaffolded by `os init` ships no ADRs, no issue tracker and none of this repo\'s ' +
`${r.scaffoldId}/${r.file} cites ${label} (${JSON.stringify(hit?.[0])}). A project ` +
`scaffolded by \`${command}\` ships no ADRs, no issue tracker and none of this repo's ` +
'scripts, so this reads as a reference the newcomer is failing to follow. State the ' +
'fact self-contained, or link a public docs page — do not delete the rationale.',
).toBeNull();
Expand All @@ -165,26 +282,26 @@ describe('rendered init templates are followable by a stranger', () => {
// ── assertion 2: the rationale survives ─────────────────────────────────
// The FACT each removed reference was carrying, matched loosely enough
// that rewording is free and deletion is not.
it.each(rendered.filter((r) => r.file === 'objectstack.config.ts').map((r) => [r.templateKey, r] as const))(
'template "%s" objectstack.config.ts still explains the protocol range',
(_templateKey, r) => {
expect(r.content, `${r.templateKey}/${r.file} must still explain why the range exists`).toMatch(
it.each(rendered.filter((r) => r.file === 'objectstack.config.ts').map((r) => [r.scaffoldId, r] as const))(
'scaffold "%s" objectstack.config.ts still explains the protocol range',
(_scaffoldId, r) => {
expect(r.content, `${r.scaffoldId}/${r.file} must still explain why the range exists`).toMatch(
/refuses this (app|plugin) at the boundary|incompatible runtime/i,
);
expect(r.content, `${r.templateKey}/${r.file} must still explain it was stamped by scaffolding`).toMatch(
expect(r.content, `${r.scaffoldId}/${r.file} must still explain it was stamped by scaffolding`).toMatch(
/stamped/i,
);
},
);

const objectFiles = rendered.filter((r) => /^src\/objects\/(?!index\.ts$)[^/]+\.ts$/.test(r.file));
it.each(objectFiles.map((r) => [`${r.templateKey}/${r.file}`, r] as const))(
it.each(objectFiles.map((r) => [`${r.scaffoldId}/${r.file}`, r] as const))(
'%s still explains the org-wide default',
(_label, r) => {
expect(r.content, `${r.templateKey}/${r.file} must still explain what OWD means`).toMatch(
expect(r.content, `${r.scaffoldId}/${r.file} must still explain what OWD means`).toMatch(
/org-wide default|OWD/i,
);
expect(r.content, `${r.templateKey}/${r.file} must still explain declaring it is required`).toMatch(
expect(r.content, `${r.scaffoldId}/${r.file} must still explain declaring it is required`).toMatch(
/required|refuses/i,
);
},
Expand All @@ -210,7 +327,7 @@ describe('rendered init templates are followable by a stranger', () => {
const urls: { where: string; url: string; route: string }[] = [];
for (const r of rendered) {
for (const m of r.content.matchAll(/https:\/\/objectstack\.ai\/docs\/([\w./-]*[\w-])/g)) {
urls.push({ where: `${r.templateKey}/${r.file}`, url: m[0], route: m[1] });
urls.push({ where: `${r.scaffoldId}/${r.file}`, url: m[0], route: m[1] });
}
}
// Non-vacuity: the rewrite puts docs links in every template on
Expand Down
Loading