Skip to content

build: reduce the network surface of the reproducible build - #193

Draft
gregorydemay wants to merge 4 commits into
mainfrom
build/robust-docker-build
Draft

build: reduce the network surface of the reproducible build#193
gregorydemay wants to merge 4 commits into
mainfrom
build/robust-docker-build

Conversation

@gregorydemay

@gregorydemay gregorydemay commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

The reproducible build on main failed on 2026-09-11 because a package download from the Ubuntu mirrors timed out. This PR shrinks what the Docker build fetches from the network and pins what remains, so that fewer things can fail and drift.

  • Base the build on the digest-pinned rust:1.93.0-bookworm image, which already ships the toolchain. The build asserts that the image's rustc matches rust-toolchain.toml. Only clang is still installed via apt (needed to compile zstd-sys for wasm32), from the dated Debian snapshot the base image was built from, so the package set no longer drifts with the live mirrors. The rustup installer is no longer downloaded.
  • Install ic-wasm as a pinned release binary verified by SHA-256 instead of compiling it from crates.io on every build.
  • Make the dependency pre-build stage actually work. It previously completed in 0.2 s because the build script was invoked without a canister and the member manifests were missing. All crates are now compiled in a layer keyed only on the Cargo manifests, so a source-only change rebuilds in about 30 s locally.
  • Drop the git dependency on the dfinity/ic monorepo (about 825 MB cloned per build) by reading the minter Wasm from a file in the integration tests, with CKSOL_MINTER_WASM_PATH as an override. Locally, build the Wasm first with ./scripts/build --cksol_minter or ./scripts/docker-build.
  • Drop jq from the build script and add a .dockerignore.

The resulting cksol_minter.wasm.gz is byte-identical to the artifact of the last successful build on main (run 32014921759), SHA-256 526f09d18a32f00b34302162bbedd8942c294a8bcacf4cad9e25aaa726c86a70.

Not included: retries for apt and cargo inside the Docker build, and a retry of the build step in CI. Those are a separate follow-up. The only retry added here is on the single curl download of the ic-wasm binary.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VpFpbJ5sgNutGGJUQ5dmkr

gregorydemay and others added 3 commits September 11, 2026 09:59
Replace the `ic-test-utilities-load-wasm` git dependency on the dfinity/ic
monorepo, which forced a full clone of that repository in every build,
with a loader that reads the Wasm from `CKSOL_MINTER_WASM_PATH` or from
the build output directories.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpFpbJ5sgNutGGJUQ5dmkr
Base the reproducible build on the digest-pinned `rust:1.93.0-bookworm`
image, which already ships the toolchain, so that only `clang` is
installed via apt and the rustup installer is no longer downloaded.
Install ic-wasm as a pinned binary verified by SHA-256 instead of
compiling it from crates.io.

The dependency pre-build stage previously did nothing because the build
script was invoked without a canister and the member manifests were
missing; it now compiles all dependencies in a layer that only depends
on the Cargo manifests. A `.dockerignore` keeps build outputs, the git
history and documentation out of the build context.

The resulting `cksol_minter.wasm.gz` is byte-identical to the one built
by the previous Dockerfile.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpFpbJ5sgNutGGJUQ5dmkr
Copilot AI lite review requested due to automatic review settings September 11, 2026 09:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Pin the clang package inputs and reconcile the documented curl retry scope.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR reduces Docker build network dependencies and replaces automatic minter Wasm compilation with pinned, locally loaded artifacts.

Changes:

  • Pins Rust and ic-wasm build inputs.
  • Improves Cargo dependency caching and removes the large git dependency.
  • Updates Wasm loading, CI configuration, documentation, and Docker context filtering.
File summaries
File Summary
scripts/build Removes jq usage.
scripts/bootstrap Uses locked ic-wasm installation dependencies.
README.md Documents the revised Wasm workflow.
integration_tests/src/lib.rs Loads minter Wasm from configurable paths.
integration_tests/Cargo.toml Removes the obsolete loader dependency.
Dockerfile Reworks the pinned build environment and dependency caching. Moderate: pin clang package versions or use a dated Debian snapshot. Nit: reconcile documented curl retry scope with implementation.
Cargo.toml Removes the git dependency and obsolete profile.
Cargo.lock Removes related transitive dependencies.
.github/workflows/ci.yml Updates Wasm artifact handling.
.dockerignore Reduces Docker build context.
Review details
  • Files reviewed: 9/10 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Dockerfile
Comment thread Dockerfile
Copilot AI review requested due to automatic review settings September 11, 2026 12:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A moderate artifact-selection issue remains unresolved.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

integration_tests/src/lib.rs:664

  • When both artifacts exist, this fixed order always selects wasms/cksol_minter.wasm.gz, even if ./scripts/build --cksol_minter has just produced a newer root artifact. After a Docker build, local source changes can therefore make the integration suite silently exercise a stale minter Wasm; avoid selecting by fixed location (for example, require the override or choose/validate the newest artifact).
  • Files reviewed: 9/10 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants