fix: make race-safe layer publishing opt-in for the e2e sandbox job - #1367
fix: make race-safe layer publishing opt-in for the e2e sandbox job#1367lucaspimentel wants to merge 2 commits into
Conversation
Automatic layer publishing predicted the next version by listing the current latest, which races with concurrent pipelines and fails when AWS assigns a higher version than predicted. Add a USE_AWS_ASSIGNED_VERSION flag that publishes once and uses the version returned by this job's own publish-layer-version call, and enable it only for the e2e sandbox publish job so the exported layer ARN always refers to that job's binary. All other auto-publishing jobs and scripts keep the previous latest-plus-one prediction and strict publish behavior, and fixed-version publishing for tags and manual versions is unchanged.
|
There was a problem hiding this comment.
🟡 Changes recommended
Uninitialized internal mode state can unintentionally bypass strict version handling.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds opt-in AWS-assigned layer versioning for e2e sandbox publishing.
Changes:
- Publishes once and uses AWS’s returned version.
- Enables this mode only for e2e sandbox jobs.
- Preserves strict version handling elsewhere.
File summaries
| File | Description |
|---|---|
.gitlab/templates/pipeline.yaml.tpl |
Enables AWS-assigned versions for e2e sandbox publishing. |
.gitlab/scripts/publish_layers.sh |
Implements publish-once version adoption. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Overview
Automatic layer publishing (
AUTOMATICALLY_BUMP_VERSION=1) predicted the next layer version by listing the current latest and adding 1, then ran a strict publish loop that fails the job when AWS assigns a higher version than predicted. Because layer versions are assigned atomically by AWS at publish time, a concurrent pipeline publishing to the same layer between the list call and the publish call makes the prediction stale, and the job fails withPublished version N is greater than the desired version N-1even though the published version holds this job's own binary.This PR adds an explicit
USE_AWS_ASSIGNED_VERSIONflag that publishes once and adopts the version AWS returns, and enables it only for thepublish layer e2e sandboxjob, which is the job that blocks serverless e2e tests when it fails. All other auto-publishing callers (sandbox, serverless_testing, self-monitoring, govcloud us1-staging-fed) keep the previous latest-plus-one prediction and strict behavior. Fixed-version publishing (prod tags, explicitVERSION) is unchanged and remains strict.For the e2e job, the exported
EXTENSION_LAYER_ARNdotenv artifact now always refers to the version this job actually published, so the downstream e2e-test trigger jobs test the binary from this pipeline.Testing
bash -n .gitlab/scripts/publish_layers.shpasses.awsCLI covering five scenarios, all passing:USE_AWS_ASSIGNED_VERSION=1(latest 698, AWS assigns 700): exits 0, onepublish-layer-versioncall, nolist-layer-versionscalls, dotenv exports the ARN of the AWS-assigned version.greater than the desired version, no dotenv (previous behavior preserved).:699ARN.VERSION=699, AWS assigns 700: still fails, no dotenv.VERSION=699, AWS assigns 699: succeeds, exports:699ARN.shellcheckfindings limited to the pre-existing SC2086/SC2059/SC2016/SC2004 classes already present onmain.