Skip to content

fix(preview): build preview images for arm64 too - #6211

Merged
nicacioliveira merged 2 commits into
mainfrom
fix/preview-images-multiarch
Aug 19, 2026
Merged

fix(preview): build preview images for arm64 too#6211
nicacioliveira merged 2 commits into
mainfrom
fix/preview-images-multiarch

Conversation

@nicacioliveira

@nicacioliveira nicacioliveira commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

A running preview died on a spot reclaim.

The studio-preview node pool schedules on amd64 and arm64 so Karpenter can take whatever spot capacity exists. The preview images were amd64 only. Karpenter replaced an r8in.large with an r7gd.large (Graviton), and every pod came back as:

exec /usr/local/bin/bun: exec format error

The release images have been multi-arch (linux/amd64 + linux/arm64) all along — previews were the outlier, on the stated grounds that "the preview cluster is amd64". The node pool that shipped alongside them does not guarantee that, and did not honour it on the first reclaim.

Why QEMU and not the release matrix

release-studio.yaml builds each architecture on a native runner and stitches the digests with buildx imagetools create. That is the right shape for the release path. Here it would mean a second job, digest artifacts and a manifest-merge step to save ~51s — by the release workflow's own numbers, emulated arm64 costs ~64s against ~13s native for the tarball-install layer. That is roughly 9% of a ~9-minute preview build. Not worth the machinery for a disposable environment.

cache-from still reads the release scopes, so the apt / useradd / duckdb layers stay cache hits.

Assertion

Nothing in a Helm render can observe an image manifest, so helm-test.yml now checks the build workflow directly: if the pool takes arm64, the build has to target it.

Alternative considered

Pinning the node pool to amd64 is one line and also fixes it — but it gives up Graviton, which is usually ~20% cheaper, and narrows the instance set on a spot-only pool, which raises the reclaim rate on exactly the workload that just proved it is sensitive to reclaims.

After merge

Existing preview images stay single-arch; they are rebuilt on the next push to their PR. #6205 is currently down for this reason and recovers on its next build.


Summary by cubic

Builds preview images for both amd64 and arm64 by building each architecture on native runners and publishing a merged manifest. Previously previews shipped amd64-only images; when pods landed on Graviton, they failed with “exec format error.”

  • .github/workflows/preview-build.yaml: split into package (tarball), docker matrix for linux/amd64 and linux/arm64 on native runners, push by digest, then publish stitches digests with docker buildx imagetools create and comments the preview URL. Image tags exist only after the manifest merge.
  • .github/workflows/helm-test.yml: assert the arm64 runner is used and that manifests are merged, since Helm renders cannot observe image manifests.

Rollout

  • No app changes. New preview builds publish multi-arch images.
  • Previews with amd64-only images must rebuild; push a new commit to the PR to trigger a build.

Written for commit faad048. Summary will update on new commits.

Review in cubic

The studio-preview node pool schedules on amd64 and arm64 so Karpenter can take
whatever spot capacity exists. The images were amd64-only. A reclaim replaced an
r8in.large with an r7gd.large (Graviton) and every pod in the running preview
died with `exec format error`.

The release images are already multi-arch; previews were the outlier, on the
grounds that 'the preview cluster is amd64' — which the node pool in the same
change set does not guarantee.

QEMU rather than the release workflow's native-runner matrix: ~64s against ~13s
for the tarball-install layer is ~9% of a ~9-minute preview build, and does not
justify a second job plus digest artifacts plus a manifest merge here.

helm-test now asserts the build targets both, since no render can see it.
Replaces the QEMU approach from the previous commit, which was wrong: the ~64s
figure it leaned on describes a single layer, not a build. Emulating the whole
arm64 image would add minutes to a build whose value is being quick enough that
a reviewer waits for it.

The release workflow already measures the answer — ~1.0 min for a native arm64
image against ~0.7 min for amd64, plus ~0.3 min to merge — so this copies its
shape: package once, build each architecture on its own runner
(ubuntu-24.04-arm is free on public repos), push by digest, and stitch the
digests into one tagged manifest list.

Wall clock is roughly unchanged, because the two architectures build in
parallel and only the manifest merge is added.
@nicacioliveira
nicacioliveira merged commit 83d1525 into main Aug 19, 2026
31 checks passed
@nicacioliveira
nicacioliveira deleted the fix/preview-images-multiarch branch August 19, 2026 00:20
nicacioliveira added a commit that referenced this pull request Aug 19, 2026
The images for this PR predate #6211 and are amd64 only, so the preview died
when a spot reclaim moved it onto a Graviton node.
nicacioliveira added a commit that referenced this pull request Aug 19, 2026
The images for this PR predate #6211 and are amd64 only, so the preview died
when a spot reclaim moved it onto a Graviton node.
nicacioliveira added a commit that referenced this pull request Aug 19, 2026
The images for this PR predate #6211 and are amd64 only, so the preview died
when a spot reclaim moved it onto a Graviton node.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant