One output is one file, named by the spec

CLAUDE.md

@@ -478,8 +478,8 @@ Without it `filter=annex` routes *everything* into the annex, analysis
 code included. `tests/test_dataset.py::test_analysis_code_stays_in_git_and_stays_writable`
 pins it against a real annex.
 
-**Manifests stay in git, deliberately.** `**/.lightcone-manifest.json` is
-exempted back out of the annex so it is readable on a clone that has
+**Manifests stay in git, deliberately.** `results/**/.*.manifest.json`
+is exempted back out of the annex so it is readable on a clone that has
 fetched no annex content at all — which is what lets `lc materialize
 --check` classify a whole project on a laptop that holds none of the
 bytes.
@@ -714,8 +714,14 @@ must not discard edits made elsewhere while the graph was running.
 ## Key Invariants (layer 2)
 
 **Two hashes, and they answer different questions** (`identity.py`).
-`definition_version = sha256(recipe ‖ canonical decisions)` is what the
-spec says an output *is*. `env_version = sha256(uv.lock bytes ‖
+`definition_version = sha256(recipe ‖ canonical decisions ‖ format)` is
+what the spec says an output *is*. The format is framed in its own right
+rather than left to arrive through the rendered `{output}` path: a recipe
+need not use that placeholder at all (`evals/tasks/snae` does not), and
+one that does not would keep its digest across a re-declared
+serialization — while the sidecar, named from the id alone, does not move
+either, so the manifest would go on describing an output at a path that
+no longer exists. `env_version = sha256(uv.lock bytes ‖
 .python-version bytes ‖ canonical install-settings JSON)` is what it ran
 under.
 
@@ -862,13 +868,15 @@ missing decision or an unresolvable input — blaming the run for a fault
 in the file, far from the line at fault. It caught three of lc's own
 test fixtures the first time it ran.
 
-**The layout is flat and path-addressed.** `results/<universe>/<output_id>/`,
-`data/` for declared inputs, and the path in a rendered recipe *is* the
-path on disk — no staging, no scratch, no relocation. The `output_id` is
-ASTRA's **qualified** id, so a sub-analysis output lands at
-`results/<universe>/<analysis>.<output>/` and one addressing scheme spans
-however deep the spec nests. Nesting is not capped: the dot separator is
-unambiguous because ASTRA ids match `^[a-z][a-z0-9_]*$`.
+**One output is one file, and the spec names it.** An output is
+`results/<universe>/<output_id>.<format>` — never a directory a recipe
+fills — and the path in a rendered recipe *is* the path on disk: no
+staging, no scratch, no relocation. `format` comes from ASTRA
+(`Output.format`, astra-spec 0.0.14), so lc composes the filename rather
+than a recipe choosing it, and the whole contents of `results/` are a
+pure function of the spec. `plan.build` **refuses** a spec whose
+executable outputs omit it, naming every one: it is only *recommended*
+until ASTRA 0.1.0, but lc has nowhere to write an output without it.
 
 **One rule names a path, and both the recipe and the run record use it**
 (`plan.declared_path`). Project-relative inside the tree, absolute
@@ -887,26 +895,46 @@ the layer makes, and saying so is the whole obligation — the same
 treatment `sdist_built` gets.
 
 **Two universes cannot share an id.** The id names a directory under
-`results/`, and the graph is keyed on `(universe_id, output_id)` — so the
+a `results/`, and the graph is keyed on `(universe_id, output_id)` — so the
 second file simply replaced the first and one universe's outputs went
 missing with nothing said. The way in is the natural one: copy
 `baseline.yaml`, edit the decisions, forget the id inside. `build`
 refuses, naming both files.
 
-**Because the path is composed, `output_dir` refuses an id that is not one
-path component.** An empty universe or output id collapses
-`results/<u>/<o>` onto a *parent* — `results/` itself, for two — and the
-worker empties that directory before running a recipe in it, so the
-consequence of an unchecked id is deleting every other universe's
-outputs. A `/`, `\`, `.` or `..` is refused for the same reason. This is
-the guard that lets the reset stay a whole-directory operation.
-
-**The reset takes the whole directory, and cannot take a named list.** A
-recipe declares an output *id*, never filenames, so there is no set of
-"expected files" to remove — and a previous run that crashed can have left
-anything in there, which would otherwise survive into this run's
-`data_version` as though the recipe had written it. What bounds the blast
-radius is the guard above, not a narrower delete.
+**Because the path is composed, `assets.output_path` refuses any part
+that is not one path component** — an empty or `..`-bearing universe or
+output id would place a file outside the tree every guard above it
+checked — and a `format` that could not be an extension (empty, a
+separator, or leading-dot, which would make the output look like its own
+sidecar). It also refuses an output id carrying a **dot**: the sidecar is
+recovered by partitioning the filename on the first one, so a dotted id
+would name a manifest for something else. ASTRA's own id grammar carries
+no dot, so this only ever fires on an id lc did not compose.
+
+**The reset takes what the output's id names, never the directory.**
+Outputs share a directory now and Dask writes them concurrently, so a
+whole-directory delete would take a neighbour's bytes. It unlinks the
+sidecar and globs `<output_id>.*`: an id cannot contain a dot, so the glob
+cannot reach a sibling, a longer id, another output's sidecar or a scope
+directory of the same name — and it *does* reach a payload left by a run
+that declared another `format`, which is what stops one orphaning.
+
+**A payload that is not a regular file fails the task.** `data_version`
+branches on `is_file()` and hashes a directory happily, so `mkdir
+{output}` — what every recipe written for the old layout does — would
+otherwise commit a well-formed digest of something that is not the
+output, plus a crate `File` with no `sha256`. One check covers absent,
+directory and wrong-name, and it is what makes exit 0 stop being
+evidence that anything was written.
+
+**The manifest is a sidecar, `.<output_id>.manifest.json`, named from the
+id alone and never the format.** So it keeps its path — and therefore its
+history — across a re-declared serialization, which is what lets
+`_foreign_write` still answer for an output whose payload path is new.
+Decomposition is safe by partitioning on the **first** dot (ids carry
+none; formats may, `tar.gz`), never `Path.stem`. The old
+`_HASH_EXCLUDE` is gone with the directory that made it necessary: the
+manifest cannot be inside the thing it describes any more.
 
 **Dask owns the ordering.** Every task is submitted with its upstream
 futures as arguments, so the dependency order, the parallelism, and the
@@ -1069,6 +1097,19 @@ raised anywhere inside astra's validation or resolution was reported as a
 bad target — a rerun misdiagnosing itself, at the one place nobody is
 watching.
 
+**Known violation, unfixed: a malformed `astra.yaml` tracebacks out of
+`lc status` and `lc materialize --check`** as a raw
+`yaml.scanner.ScannerError`. `plan._validate` is the gate that turns a
+bad spec into ASTRA's own errors, but it only catches what *validation*
+reports — a spec that does not parse raises inside
+`validate_analysis_schema` → `astra.helpers.load_yaml` before there is
+anything to validate. (`lc materialize` usually masks it by refusing a
+dirty tree first, which is not a fix.) The shape of the fix is a
+`yaml.YAMLError` catch in `_validate` naming the file and the line, the
+same way the validator's own errors are rendered. Pre-existing and
+independent of the one-output-one-file change; written down here because
+it contradicts the invariant directly above it.
+
 **`git_sha` in a manifest is the commit the run *started* at**, not the
 commit the run went on to create. It is the code that produced the output.
 A test that reads `dataset.head()` after materializing and expects a match
@@ -1208,20 +1249,21 @@ impurity lives in policy construction, once.)
 
 **There is one policy, `exec_policy`, and it differs between a recipe
 and a probe by one keyword.** The tree is read-only apart from the
-in-tree write scope: a recipe's is **its own output directory**
-(`output_dir=`, handed down from the worker's task), a probe's is
-`results/` whole, because a probe has no output id. This narrowing is
-the hardening pass reversing an earlier decision (see Recorded
-decisions): it exists as leak *prevention*, closing the cross-write
-residue — a concurrent task landing bytes in a sibling's directory
-before the sibling hashes produced a manifest that was self-consistent
-and wrong, which no checksum could ever see, so prevention was the only
-possible fix. The probe→recipe promise ("a command that works under
-`lc run` works as a recipe") now excludes exactly the commands that
-write outside their own output directory — which is the accident being
-prevented, not a loophole in the promise. Integrity-answering is still
-`data_version`'s job alone; the sandbox prevents the write, it does not
-attest the bytes.
+in-tree write scope: a recipe's is **the directory its output lands in**
+(`write_dir=`, handed down from the worker's task), a probe's is the
+project's own `results/` whole, because a probe has no analysis node.
+
+That is a *directory* and not the output file, by mechanism rather than
+by choice: `policy._declared`/`_existing` drop paths that do not exist,
+so a file scope would leave a recipe with no in-tree write at all, and
+the Landlock shim masks a non-directory grant to `_FILE_ONLY_BITS`,
+stripping `MAKE_REG` — a file grant could not create the file. So the
+one-output-one-file change reverses the hardening pass's narrowing:
+outputs declared side by side are mutually writable again. Recorded, not
+papered over — what answers whether an output's bytes are its own is
+`data_version`, never the sandbox. What the scope still excludes is
+every *other* results tree: another universe, and another analysis's
+own.
 
 **`cluster_for_run()` is the seam, and it is two methods wide.**
 `submit(fn, *args, key=…)` and `completed(handles)`. That is all the
@@ -1878,7 +1920,13 @@ JSON-LD drops on expansion — the pre-rebuild exporter's silent failure.
 The committed archive is one entity, `["File", "ContainerImage"]`,
 identity (`sha256` = config-blob id) and payload together.
 
-**A published `sha256` is always a raw digest an outsider can verify** —
+**A published `sha256` is always a raw digest an outsider can verify**, and
+since the one-output-one-file change the manifest agrees with it: an
+output is a file, `data_version` drops the `file:` frame, and the
+manifest's digest, the crate's `sha256` and `sha256sum` are one number.
+Only directory-valued declared inputs stay framed (`dir:`), which is
+enough to keep the two from colliding and is the one entry in a manifest
+no standard tool can check. Historically —
 since the hardening pass, never lc's framed hash. Every file in an
 output directory is a `File` under its dataset's `hasPart`, with
 `sha256` and `contentSize` parsed from its SHA256E annex key
@@ -2141,6 +2189,50 @@ unlinks before writing; a new tampering test should too.
     materialize runs for hours) arrives with the submission-model
     venue.
 
+### Recorded decision: one output, one file (post-hardening)
+
+`Output.format` landed in astra-spec 0.0.14 / astra-tools 0.2.17, and lc
+took it as the missing half of a long-standing asymmetry: `Input.source`
+always declared a path with an extension, while an output declared
+nothing and `{output}` was documented as "the path the artifact will be
+written to" — a path the schema never defined. What changed, and why:
+
+- **An output is one file, not a directory.** The multiplicity was born
+  the moment a recipe was handed a directory and picked a name inside it.
+  There is no directory now, so the natural idiom (`savefig(args.out)`)
+  produces exactly one file at exactly the declared path. The cost,
+  stated: a figure plus its data, a zarr store, a checkpoint directory
+  must become separate outputs or a container format (`.h5`, `.zip`,
+  `.tar.gz`). No flag re-opens it.
+- **The filename contract between producer and consumer is gone.**
+  `{inputs.X}` renders to the upstream's *file*, so nothing has to know
+  what is inside a directory it was handed.
+- **`.gitattributes` is convergence's to get right, not a run's.**
+  `dataset.save` passes `annex.dotfiles=true`, so the sidecar's leading
+  dot decides nothing and the attributes decide everything — annexed, a
+  run is green locally while every clone reports the project as never
+  materialized. A `check-attr` preflight in `materialize` was written and
+  removed: `lc init` owns that file, its repair appends the exemption and
+  its disorder guard blocks a file it cannot fix, and a second answer in
+  the run path is one more place for the two to disagree. The residue,
+  stated: a project scaffolded before the exemption existed annexes its
+  manifests until someone converges it — which this change already
+  requires, since every output needs a `format:` added by hand.
+- **The orphan walk is a set difference, never a reconstruction.**
+  Expected sidecars come from `Task.manifest_path`; present ones from
+  `git ls-files`, which still finds a manifest whose output the spec has
+  just dropped — the edit that orphans it is the same edit that would
+  drop it from anything derived from the graph.
+- **No migration, by decision.** Nothing looks for `.lightcone-manifest.json`
+  and nothing detects directory-shaped outputs from a previous engine;
+  they are invisible to every walk. `git rm -r results/` and
+  re-materialize.
+- **A nested spec is not buildable.** lc materializes a flat analysis:
+  ASTRA qualifies an output declared in a sub-analysis as `<a>.<b>`, and
+  `output_path` refuses a dotted id because the sidecar could not be told
+  from the payload. Supporting nesting means deciding where a nested
+  output's file goes, which is its own change (issues #201, #202).
+
 ### Recorded deviations from the spec
 
 - **No `AGENTS.md` scaffolding** (spec §2 calls for an agent notes

docs/api/assets.md

@@ -16,8 +16,9 @@ Source: `src/lightcone/engine/assets.py`.
 | `Verdict.calls_for_a_remake(refresh=)` | The one place a state becomes an action: `stale` always, `behind` only when asked. |
 | `data_version(path)` | Content hash of a directory or file — computed in the worker, before anything is annexed. |
 | `Versions` | Per-run memo so a shared declared input hashes once, not once per dependent. |
-| `read(dir)` / `write(...)` | The manifest, `.lightcone-manifest.json`. |
-| `output_dir(root, u, o)` | The path, guarded: an id that is not one path component is refused — this guard is what lets the worker's reset stay a whole-directory delete. |
+| `read(sidecar)` / `write(...)` | The manifest, `.<output_id>.manifest.json`. Both take the sidecar's own path, so a caller holding an output path has to say `manifest_path` out loud. |
+| `output_path(root, u, id, fmt)` | The output's file, guarded: any part that is not a single path component is refused, and so is a format that could not be an extension. |
+| `manifest_path(output)` | The sidecar beside it, named from the id alone — so it keeps its path, and its history, across a re-declared format. |
 | `ContentNotFetchedError` | An annexed file whose content is not in this clone, in either shape it takes. |
 
 ## What must stay true

docs/api/container.md

@@ -46,7 +46,8 @@ Sources: `src/lightcone/engine/image.py`,
   after the graph resolves** — a refusal over a typo must not cost a
   minutes-long build, and `dataset.save` commits the whole index.
 - **The mount table is the mechanism** (`sandbox/oci.py`): project
-  `:ro`, `results/` `:rw`, declared inputs `:ro`, private HOME,
+  `:ro`, the write scope `:rw` — the directory a recipe's output lands
+  in, or `results/` for a probe — declared inputs `:ro`, private HOME,
   `--tmpfs /tmp`, over a `--read-only` rootfs — without that flag a
   stray write *succeeds* into the ephemeral layer and vanishes while
   the attestation claims `fs: declared`. Mounts are resolved source,

docs/api/crate.md

@@ -39,6 +39,10 @@ Source: `src/lightcone/engine/crate.py` (converged by
 - **The `Person` is the author of the output's *saving* commit** (via
   `writer`), never the manifest's `git_sha` — that is the commit the
   run *started* at and can be someone else's.
+- **An output is a `File`, not a `Dataset` of parts.** It is one file,
+  so there is one annex key to look up and one `sha256` to publish —
+  the same number its manifest records as `data_version`, and the one
+  `sha256sum` prints.
 - **The manifest is not transliterated.** `env_version`,
   `definition_version` and `hermeticity` get no invented schema.org
   spelling — the manifest itself is in the crate as a `File`,

docs/api/dataset.md

@@ -17,7 +17,7 @@ Source: `src/lightcone/engine/dataset.py` (+
 | `restore(root, paths)` | `git clean` always; `git checkout HEAD --` only when HEAD has the path. Never `-- .`. |
 | `status(root)` | The dirty question, scoped to the project (`-- .`, prefix-stripped) so a project inside a larger repository works. |
 | `head(root)` | The commit a run started at — read once per run, by the driver. |
-| `last_writer(root, dir)` | Who last touched an output's directory — the foreign-write question. Answers "cannot say" as empty, never an error. |
+| `last_writer(root, *paths)` | Who last touched an output or its manifest — the foreign-write question. Answers "cannot say" as empty, never an error. |
 | `require_committer(root)` | Refuses a repository with no git identity, before any recipe spends time. Asked as `git var`, the question a commit itself asks. |
 | `dataset_id(root)` | The DataLad dataset UUID, read via `git config -f`. |
 | `set_annex_filter_required(root)` | Set `filter.annex.required=true`, so a `git add` that cannot reach git-annex fails loudly instead of staging raw bytes. |

docs/api/plan.md

@@ -32,7 +32,7 @@ Source: `src/lightcone/engine/plan.py`.
   resolution answers what a *valid* spec means and does not re-check
   that it is one.
 - **The layout is flat and path-addressed.**
-  `results/<universe>/<qualified_output_id>/`, and the path in a
+  `results/<universe>/<id>.<format>`, and the path in a
   rendered recipe *is* the path on disk — no staging, no relocation.
 - **`declared_path` is lexical, never `resolve()`d.** A declared input
   under `data/` is an annex symlink; resolving it writes