Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .anvil.lock
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
version = 1
tool = "anvil"
tool_version = "0.7.0"
catalog_checksum = "sha256:3415a24c3e1b7216ecfd37ecfda02b4e6039851b70f07e45a676597254874191"
catalog_checksum = "sha256:1e071bf7b0c197d567567501fda541c074e267ff51c62c5b3233f64bb00c695c"

[[file]]
path = ".anvil/container/Dockerfile.dockerignore"
checksum = "sha256:9c7906c20415ca3b832afb075c21e79f191ef14ee2fb4b6a5a95394e8b2153c1"
checksum = "sha256:427447e66dcebc14fe00bc4c6d8c8e8c972b16b7ba0fce1fb3a2bf39cfe4cc30"

[[file]]
path = ".github/actions/anvil-impact/action.yml"
Expand Down Expand Up @@ -173,7 +173,7 @@ checksum = "sha256:6efd7378a2cd0f5d86519bd32fd86f2055a60191187dd77a8842b374b8eb7

[[file]]
path = "justfiles/anvil/container.just"
checksum = "sha256:3363992c6c006c649eae3732b2a7b04c2a4925eff369179b128d43380e2800d4"
checksum = "sha256:fd9b76f6fffb8d33ee6f6f4039eae08838f2027fd80c342412db7c49154710e6"

[[file]]
path = "justfiles/anvil/groups/pr-fast.just"
Expand Down Expand Up @@ -233,7 +233,7 @@ checksum = "sha256:00453a12cbb34811ee6a2c083dade5f6198575e3b0610f49e4743366326cd

[[file]]
path = "justfiles/anvil/tools.just"
checksum = "sha256:48b887481a7eb8dc7615367347c93b8a02dd7baafb0ef80ece883f03600285dd"
checksum = "sha256:a4dcb0de11f57187f88cc1c27e0263391a74b53b09ccd27428f0d07c022bd524"

[[file]]
path = "justfiles/anvil/versions.just"
Expand All @@ -257,7 +257,7 @@ checksum = "sha256:7b409a9b560c214e10b50f74330fb6f8c0c12c3d83494e0dcf016f2411b50
[[region]]
host = ".anvil/container/Dockerfile"
id = "anvil-container-setup"
checksum = "sha256:3788845ea3d4c483917954ddf9bccf918bf69ab7c64bbf559baf59d5781c444f"
checksum = "sha256:46d1c7acf41ce4a50444be9db1b1510fcf704127877d18bfbcc5b79f99c6e737"

[[region]]
host = ".anvil/container/Dockerfile"
Expand Down
23 changes: 21 additions & 2 deletions .anvil/container/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ RUN curl -fsSLo /tmp/cargo-binstall.tgz \
# <<< anvil-managed: anvil-container-tools

# >>> anvil-managed: anvil-container-setup
# The context is copied whole rather than input by input, because one of the
# inputs is optional: a repository that pins its compiler by other means owns no
# root toolchain file, and a `COPY` of a path that may not exist is not portable
# across the engines anvil supports. `.anvil/container/` rides along as a
# consequence; the image never reads it.
#
# The whole recipe tree is copied because `just` parses it to reach the install
# recipes.
#
Expand All @@ -94,11 +100,24 @@ RUN curl -fsSLo /tmp/cargo-binstall.tgz \
# volume over each, and an engine seeds a new volume from the image path it
# covers; a path that does not exist seeds as root-owned 0755, which the
# `--user` mapping cannot write, so the first cargo fetch fails with EACCES.
# The root manifest is admitted for the MSRV, the one version `anvil-setup`
# installs that is declared in repository source rather than pinned in
# `versions.just`. `/opt/anvil` is the root the recipes already resolve against,
# so the resolver reads it there with no container-specific path. The workspace
# members it names are not admitted and are not needed: the one path that reads
# them is workspace MSRV validation, which hangs off the `-validate-prereqs`
# recipes, and `anvil-setup` depends on none of them.
#
# It is deleted once the setup has read it, so it does not survive into the
# finished image. It is the one thing the context carries that the tag does not
# hash -- the declared MSRV is hashed instead, so an unrelated dependency edit
# computes the same tag -- and a manifest left in place would make that tag name
# two different filesystems.
WORKDIR /opt/anvil
COPY justfiles ./justfiles
COPY rust-toolchain.toml ./
COPY . ./
RUN printf "import 'justfiles/anvil/mod.just'\n" > Justfile \
&& just anvil-setup binstall \
&& rm -f Cargo.toml \
&& rm -rf "${CARGO_HOME}/registry/cache" "${CARGO_HOME}/registry/src" \
&& rm -f "${CARGO_HOME}/credentials" "${CARGO_HOME}/credentials.toml" "${HOME:-/root}/.netrc" /root/.netrc \
&& mkdir -p "${CARGO_HOME}/registry" "${CARGO_HOME}/git" \
Expand Down
19 changes: 16 additions & 3 deletions .anvil/container/Dockerfile.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@
# `.dockerignore`, so this scopes the exec-image build context without the
# repository having to own a root ignore file or having one silently overridden.
#
# The build context is the repository root but the image only needs two things.
# Excluding everything else keeps a cold build from streaming the whole
# worktree (and every stale `target/`) to the daemon.
# The build context is the repository root but the image needs a small, named
# set out of it. Excluding everything else keeps a cold build from streaming the
# whole worktree (and every stale `target/`) to the daemon -- and, because the
# setup region copies the context whole, this file decides what the image
# contains.
#
# The context is narrowed to `justfiles/anvil/` rather than all of `justfiles/`
# so that a cold build does not stream unrelated trees to the daemon. The
Expand All @@ -25,6 +27,15 @@
# the image tag digests, so what the context admits and what the tag covers stay
# the same set -- including the `.anvil-proposed` siblings both exclude, which
# are anvil's review artifacts rather than build inputs.
#
# The root `Cargo.toml` is admitted for the declared MSRV, the only build input
# that lives in repository source rather than in the recipe tree. The workspace
# members it names stay out: they are a checkout, and the image is not one.
#
# Neither toolchain-file spelling is required, and both are admitted: a
# repository that pins by other means builds an image whose compiler is the
# declared MSRV, and admitting only the TOML would give one that pins with the
# extensionless file an image that silently disagreed with its own checkout.
*
!justfiles
justfiles/*
Expand All @@ -35,3 +46,5 @@ justfiles/anvil/**/*.anvil-proposed
!.anvil/container
.anvil/container/**/*.anvil-proposed
!rust-toolchain.toml
!rust-toolchain
!Cargo.toml
18 changes: 11 additions & 7 deletions crates/cargo-anvil/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,8 @@ pin the first image’s tools over every later one.
through `wsl.exe` when it finds none on `PATH` and translates repository
paths with `wslpath`.
* `just` and `PowerShell` Core (`pwsh`) on the host.
* A repository-owned `rust-toolchain.toml`.
* A declared root MSRV. A repository-owned `rust-toolchain` file, in either
spelling, is honoured where it exists but is not required.

Docker is supported; Podman works on a best-effort basis, with two
documented gaps on Windows. The image is pinned to `linux/amd64`, so on
Expand All @@ -166,8 +167,9 @@ ARM64 hosts it is emulated and is substantially slower.
#### Image identity

The tag *is* a SHA-256 digest over the inputs that define the image:
everything under `.anvil/container/`, `rust-toolchain.toml`, and the whole
generated `justfiles/anvil/` tree. The container directory is walked rather
everything under `.anvil/container/`, a root `rust-toolchain` file where the
repository owns one, and the whole generated `justfiles/anvil/` tree. The
container directory is walked rather
than named file by file, because the Dockerfile is composed and a
repository can `COPY` a certificate or an install script it places there.
The recipe tree is included in
Expand Down Expand Up @@ -257,9 +259,11 @@ those pins at the moment of the edit, which is why the gaps exist.

A downstream catalog that needs a different base OS for every repository it
manages replaces the base and tool regions instead, inheriting the catalog
install and the entry contract. A replacement that copies more of the tree
must replace the ignore file with it, since the build context admits only
`justfiles/anvil/`, `.anvil/container/` and `rust-toolchain.toml`. See
install and the entry contract. A replacement that needs more of the tree
must replace the ignore file with it: the setup region copies the context
whole, so that file decides what the image contains, which today is
`justfiles/anvil/`, `.anvil/container/`, a root `rust-toolchain` file and
the root `Cargo.toml`. See
[`artifacts::container`][__link1] and the design document for the full contract, the
host setup for each engine, and the known limitations.

Expand Down Expand Up @@ -491,7 +495,7 @@ And `docs/verification.md` for the continuous-validation strategy.
This crate was developed as part of <a href="../..">The Oxidizer Project</a>. Browse this crate's <a href="https://github.com/microsoft/ox-tools/tree/main/crates/cargo-anvil">source code</a>.
</sub>

[__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjNhdIQbFhzZ8rzWNNYbuRaDSGWynFgbH4PMdoT7GNcbVwNPtPjAhvFhYvRhcoQbVqn03OrTnSYblGjeKgXeGVgb6z3iwQiK18Abc5kLxsXto9xhZIGDa2NhcmdvLWFudmlsZTAuNy4wa2NhcmdvX2Fudmls
[__cargo_doc2readme_dependencies_info]: ggGmYW0CYXZlMC43LjNhdIQbFhzZ8rzWNNYbuRaDSGWynFgbH4PMdoT7GNcbVwNPtPjAhvFhYvRhcoQb1SwbqKADGLgb2SUB5pyIW5AbZYWAp-o-cjIb3p-idO-iZi1hZIGDa2NhcmdvLWFudmlsZTAuNy4wa2NhcmdvX2Fudmls
[__link0]: https://crates.io/crates/cargo-delta
[__link1]: https://docs.rs/cargo-anvil/0.7.0/cargo_anvil/?search=artifacts::container
[__link10]: https://docs.rs/cargo-anvil/0.7.0/cargo_anvil/?search=artifacts
Expand Down
72 changes: 55 additions & 17 deletions crates/cargo-anvil/docs/design/containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,10 @@ Both are addressed by executing the recipe unchanged inside an image built from
bodies are identical in either mode, and cloud workflows are unaffected: they run the same recipes natively on their
own agents. The image is pinned to resemble that environment, not to reproduce it.

Image construction has a deliberately stricter compiler contract than native execution: the repository must own
`rust-toolchain.toml`. The build context admits that file but not `rust-toolchain`, and the build does not inherit the
host's `RUSTUP_TOOLCHAIN`; rustup therefore selects the image compiler from the repository-owned TOML file.
Image construction does not require the repository to own a `rust-toolchain.toml`. Where one exists — either
spelling — the context admits it and rustup selects the image compiler from it, exactly as it would on a host. Where a
repository pins its compiler by other means and owns neither, the image compiler is the declared root MSRV, which is
again what a host resolves there. The build does not inherit the host's `RUSTUP_TOOLCHAIN` in either case.

## 2. Command surface

Expand Down Expand Up @@ -133,7 +134,7 @@ never read, rewritten or reordered.
| `anvil-container-base-image` | `ARG BASE_IMAGE`, pinned to a digest. | A second `ARG BASE_IMAGE=…` to build on a different base. |
| `anvil-container-base` | `FROM`, the version pins for `pwsh`, `just`, `rustup` and `cargo-binstall`, and the `ENV` block. | Anything the first network access needs: a root CA, `http_proxy`, an internal package mirror. |
| `anvil-container-tools` | System packages and those four tools. | Libraries a catalog tool needs to compile, for tools `binstall` has no prebuilt binary for. |
| `anvil-container-setup` | `COPY` of the recipe tree, then `just anvil-setup`. | Anything the repository's own checks need at run time. |
| `anvil-container-setup` | `COPY` of the recipe tree and the root manifest, then `just anvil-setup`. | Anything the repository's own checks need at run time. |
| `anvil-container-entry` | `ANVIL_IN_CONTAINER`, `WORKDIR`, `CMD`. | — |

Each gap sits at the only point in the build where its kind of addition works: a certificate has to land before the
Expand Down Expand Up @@ -185,12 +186,33 @@ The image installs its tools by running `just anvil-setup`, the same recipe the
generated pins. There is no second tool list to keep synchronized, and consequently a tool-pin change renames the
image (§4.1).

`Dockerfile.dockerignore` scopes the build context to `justfiles/anvil/`, `.anvil/container/` and
`rust-toolchain.toml`, denying everything else. The recipe tree is copied whole because `just` has to parse it to run
`anvil-setup`, and it is hashed whole (§4). `.anvil/container/` is admitted so a gap can `COPY` a file placed beside
the Dockerfile; anvil's own `.anvil-proposed` review artifacts are excluded from both the context and the digest.
BuildKit reads `<dockerfile>.dockerignore` in preference to a root `.dockerignore`, so the repository neither needs to
own a root ignore file nor can have one silently override this.
One version that recipe needs is not in the pins. `anvil-msrv-test-setup` installs the toolchain named by the
repository's declared MSRV, which lives in the root `Cargo.toml`. That manifest is therefore admitted to the context
and copied to `/opt/anvil`, the root the recipes already resolve against, so the MSRV resolver reads it there with no
container-specific path in it.

The workspace members it names are not admitted: they are a checkout, and the image is not one. The one path that
would need them is workspace MSRV validation, which the image never reaches: it hangs off
`anvil-tool-rustc-validate-prereqs`, and no `-setup` recipe depends on a `-validate-prereqs` recipe. Inside a running
container that validation does execute, against `/workspace` — a real checkout, with its members.

The manifest is deleted once the setup has read it, so it is in the build context but not in the finished image. That
keeps the tag honest: it hashes the declared MSRV rather than the file, so a dependency edit computes the same tag,
and nothing is left behind for that tag to misdescribe.

The setup region copies the context whole rather than naming each input, because one input is optional. A repository
that pins its compiler by other means owns no root toolchain file, and a `COPY` of a path that may not exist is not
portable across the engines anvil supports, so naming the file would leave exactly those repositories unable to build
an image at all. Deferring to the ignore file costs no breadth — it already scopes the context to precisely the image's
inputs — and makes what the context admits and what the image contains the same set by construction. `.anvil/container/`
rides along with it; that is the committed input a gap `COPY`s from, and the image never runs it.

`Dockerfile.dockerignore` scopes the build context to `justfiles/anvil/`, `.anvil/container/`, a root toolchain file in
either spelling, and the root `Cargo.toml`, denying everything else. The recipe tree is copied whole because `just` has
to parse it to run `anvil-setup`, and it is hashed whole (§4). `.anvil/container/` is admitted so a gap can `COPY` a
file placed beside the Dockerfile; anvil's own `.anvil-proposed` review artifacts are excluded from both the context
and the digest. BuildKit reads `<dockerfile>.dockerignore` in preference to a root `.dockerignore`, so the repository
neither needs to own a root ignore file nor can have one silently override this.

## 4. Image identity

Expand All @@ -202,8 +224,9 @@ define the image. The name derives from the repository directory (§5.1).
| Input | Hashed |
| --- | --- |
| every file under `.anvil/container/` | always |
| `rust-toolchain.toml` | always |
| `rust-toolchain.toml` or `rust-toolchain` | when the repository owns one |
| every file under `justfiles/anvil/` | always |
| the declared root MSRV | always |

`.anvil/container/` is hashed by walking it, not as a fixed list of three known files. The Dockerfile is composed, so a
repository can `COPY` something from one of its gaps — a root CA, an install script, a patch — and a downstream
Expand All @@ -212,6 +235,11 @@ change the image under a reference that already resolves, which is the hole the
Dockerfile is still a hard error, checked by name: the walk alone would let it contribute nothing and yield a confident
tag for an image that cannot be built.

The root toolchain file is the one input whose absence is not an error. It is discovered rather than required, in both
spellings, and a repository that owns none simply contributes one fewer record to the digest — a state distinct from
owning one, so the two cannot share a tag. The ignore file is still required by name, for the reason above: it
contributes nothing to the digest but decides what the context, and therefore the image, contains.

The recipe tree is hashed in full. `just anvil-setup` reaches the install recipes through the tier, group and check
recipes, so the routing decides *whether* a tool is installed just as surely as `tools.just` decides *how*: dropping an
`anvil-<check>-setup` dependency from a group changes the installed set while `tools.just` and `versions.just` stay
Expand All @@ -225,6 +253,13 @@ into the build.
The cost is that editing any recipe renames the image and the next run rebuilds it. That is the correct trade: a tag
that can name contents the image does not have makes every guarantee below meaningless.

The declared root MSRV is the one input that is not a file. The image installs that toolchain, so raising it changes
what the image contains and must rename it. The digest takes the resolved value rather than the manifest declaring it:
`Cargo.toml` is the busiest file in a workspace while `rust-version` moves perhaps once, so hashing the file would
rename the image, and oblige a publisher to rebuild and republish, for a stream of edits that cannot alter a byte the
image contains. That last clause is what the setup region's `rm` buys: the manifest is read and deleted, so an edit
unrelated to `rust-version` computes the same tag for a filesystem that really is identical.

The hook file's **content** is an input, since it determines what the build installs. Its **output** is deliberately
excluded: a credential must never influence a tag.

Expand Down Expand Up @@ -395,7 +430,8 @@ image, executes the requested command directly instead of launching another cont
## 6. Engines and host setup

anvil installs nothing and manages no virtual machine. Beyond the engine, the host needs `just` and PowerShell Core
(`pwsh`), which every generated recipe requires, and the repository must own a `rust-toolchain.toml`.
(`pwsh`), which every generated recipe requires. The repository needs a declared root MSRV; a `rust-toolchain.toml` is
honoured where it exists but is not required (§1).

| | Docker | Podman |
| --- | --- | --- |
Expand Down Expand Up @@ -611,9 +647,10 @@ Replacing a *region* rather than the whole file is what makes a downstream catal
private-environment catalog rewrites the base and tool layers and nothing else. Replacing `dockerfile_setup()` reintroduces the
second tool list the design exists to avoid, and is almost never right.

**A replacement must keep the ignore file in step.** A region that `COPY`s anything outside `justfiles/anvil/`,
`.anvil/container/` and `rust-toolchain.toml` must also replace `artifacts::container::dockerignore()` (§3), or the
added paths never reach the build context and the build fails on a missing file.
**A replacement must keep the ignore file in step.** The setup region `COPY`s the context whole, so the ignore file is
what decides the image's contents. A region that needs anything outside `justfiles/anvil/`, `.anvil/container/`, a root
toolchain file and the root `Cargo.toml` must also replace `artifacts::container::dockerignore()` (§3), or the added
paths never reach the build context.

**Anything extra it copies is digested, provided it lives under `.anvil/container/`.** The hashed set is that whole
directory (§4.1), so an installer script, a config file or a certificate placed beside the Dockerfile is an input:
Expand All @@ -625,8 +662,9 @@ manual `ANVIL_CONTAINER_NO_CACHE=1`.
catalog-owned files. The reason is legibility rather than identity: the directory is the recipe tree, `just` parses
every file the image copies, and a catalog that hides an installer script there makes the tool set harder to reason
about than one that keeps it in `.anvil/`. Identity is safe either way, because the digest covers every file the build
context admits (§4.1), not only the recipes — a repository that adds a non-recipe file by hand still renames the tag
when it edits it.
context admits and the image keeps (§4.1), not only the recipes — a repository that adds a non-recipe file by hand
still renames the tag when it edits it. The root `Cargo.toml` is the one admitted file the digest does not cover as
bytes, and it is also the one the setup deletes once read, so it is in no image for the tag to misdescribe.

A fork inherits everything else: the recipes, the identity scheme, the cache volumes, the mounts, and the re-entry
guard. A different base OS with a different toolchain source is two region replacements plus one hook.
Expand Down
Loading
Loading