Skip to content

SRE-1003: Let a deploy catalog entry choose its ECR architecture #9540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
TimDiekmann wants to merge 2 commits into
base: main
Choose a base branch
from
+37 −17
Draft

SRE-1003: Let a deploy catalog entry choose its ECR architecture #9540

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bacc191
SRE-1003: Let a deploy catalog entry choose its ECR architecture
TimDiekmann Sep 4, 2026
57fbb92
SRE-1003: Let a catalog entry list the architectures it builds
TimDiekmann Sep 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 37 additions & 17 deletions .github/workflows/deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,11 @@ jobs:
# context is their own directory, so a git diff of `paths` is a complete
# change signal. `ecs` lists the ECS task definitions to redeploy when the
# image publishes; one app may drive several (e.g. `graph` → graph +
# graph-admin + type-fetcher).
# graph-admin + type-fetcher). `arches` lists what to build; it defaults
# to both for GHCR services (multi-arch manifest) and to arm64 otherwise.
# ECR holds one architecture per repository (Inspector does not scan
# manifest lists): the service's only arch when it builds one, else
# arm64, unless `ecr_arch` says otherwise.
- name: Determine changed packages
id: packages
env:
Expand Down Expand Up @@ -190,18 +194,32 @@ jobs:
)]' <<<"$CATALOG")
fi

# Build matrix: expand each affected entry to its arches. Skip amd64
# for services that don't publish to GHCR (ECR is arm64-only) and
# also on PR events (arch divergence in Rust/Node is rare;
# merge_group still validates amd64 before main).
# Resolve each entry's arches and ECR arch (see the catalog comment),
# then fail loudly if the ECR arch is not among the arches built —
# the ECR push would otherwise silently never happen.
RESOLVED_CATALOG=$(jq -c '
[.[]
| (.arches // (if (.push | index("ghcr")) then ["arm64", "amd64"] else ["arm64"] end)) as $arches
| (.ecr_arch // (if ($arches | length) == 1 then $arches[0] else "arm64" end)) as $ecr_arch
| . + {arches: $arches, ecr_arch: $ecr_arch}
]' <<<"$AFFECTED_CATALOG")
BAD_ECR_ARCH=$(jq -c '[.[] | select((.push | index("ecr")) and ((.ecr_arch as $a | .arches | index($a)) | not)) | .service]' <<<"$RESOLVED_CATALOG")
if [[ "$(jq length <<<"$BAD_ECR_ARCH")" -gt 0 ]]; then
echo "::error ::ecr_arch is not among the arches built for: $BAD_ECR_ARCH"
exit 1
fi

# Build matrix: expand each entry to its arches. On PR events amd64 is
# skipped where arm64 covers the build (arch divergence in Rust/Node
# is rare; merge_group still validates amd64 before main), but never
# for a service whose only arch it is.
BUILD_MATRIX=$(jq -c --argjson skip_amd64 "$SKIP_AMD64" '
[.[] as $e
| ({arch: "arm64"}, {arch: "amd64"})
| $e + .
| select(($e.push | index("ghcr")) or .arch == "arm64")
| select($skip_amd64 != true or .arch != "amd64")
| del(.package, .ecs, .paths)
] | { include: . }' <<<"$AFFECTED_CATALOG")
| $e.arches[]
| $e + {arch: .}
| select($skip_amd64 != true or .arch != "amd64" or ($e.arches | length) == 1)
| del(.package, .ecs, .paths, .arches)
] | { include: . }' <<<"$RESOLVED_CATALOG")

# Manifest matrix: services that publish multi-arch to GHCR.
MANIFEST_MATRIX=$(jq -c '
Expand All @@ -213,8 +231,8 @@ jobs:
[.[] | .ecs[]] | { include: . }
' <<<"$AFFECTED_CATALOG")

# Staging matrix: services whose arm64 image is pushed to ECR and
# tagged :staging (one guard run per such service).
# Staging matrix: services whose image is pushed to ECR and tagged
# :staging (one guard run per such service).
STAGING_MATRIX=$(jq -c '
map(select(.push | index("ecr"))) | { service: [.[].service] }
' <<<"$AFFECTED_CATALOG")
Expand Down Expand Up @@ -342,6 +360,7 @@ jobs:
env:
PUSH_LIST: ${{ join(matrix.push, ' ') }}
ARCH: ${{ matrix.arch }}
ECR_ARCH: ${{ matrix.ecr_arch }}
REF: ${{ github.ref }}
EVENT: ${{ github.event_name }}
SERVICE: ${{ matrix.service }}
Expand All @@ -358,7 +377,7 @@ jobs:
[[ "$REF" == refs/heads/main ]] && is_main=true

push_ecr=false
if [[ " $PUSH_LIST " == *" ecr "* && "$ARCH" == arm64 && "$is_main" == true && "$is_push_event" == true ]]; then
if [[ " $PUSH_LIST " == *" ecr "* && "$ARCH" == "$ECR_ARCH" && "$is_main" == true && "$is_push_event" == true ]]; then
push_ecr=true
fi
push_ghcr=false
Expand All @@ -378,8 +397,9 @@ jobs:
echo "push_ghcr=$push_ghcr"
echo "push=$push"

# ECR is single-arch (arm64), tag-based. Only immutable tags here
# (:sha / :run); the mutable :staging tag is set by the `stage` job.
# ECR is single-arch (the entry's ecr_arch), tag-based. Only
# immutable tags here (:sha / :run); the mutable :staging tag is
# set by the `stage` job.
echo "ecr_tags<<EOF"
if $push_ecr; then
echo "$ecr_host/$ECR_REPO/$SERVICE:sha-$SHA"
Expand Down Expand Up @@ -472,7 +492,7 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1

# ECR images are single-arch (arm64). Advance :staging onto this commit's
# ECR images are single-arch. Advance :staging onto this commit's
# immutable sha tag only if it is newer than the one currently tagged.
- name: Advance :staging if newer
uses: $/.github/actions/tag-if-newer
Expand Down
Loading