fix(deploy): stop the deploy script rewriting itself mid-run - #5
Merged
Conversation
The first SSH deploy failed twice, for two separate reasons. 1. exit 126, "Permission denied". All three files in scripts/ are mode 100644 in git. The Azure path invoked the script as "bash deploy.sh" so the missing execute bit never mattered; the forced-command wrapper exec's it directly. The script also chmod +x's itself, which it cannot do if it cannot run. Sets the execute bit in the index for all three. 2. The rerun then executed the *old* script - it printed "triggered keyless deployment via Azure Run Command" and did an unconditional docker login - despite the checkout having been updated moments earlier. deploy.sh replaces its own file with "git reset --hard", and bash reads a script incrementally by byte offset, so execution continues from the new file at the old offset. Any deploy that changes deploy.sh runs a blend of both versions. Splits the script into two stages. Stage 1 syncs the checkout and re-execs; stage 2 does the deploy and is what the fresh copy runs. COMMIT_SHA is exported across the re-exec and still takes precedence over the stale value in .env.
All three were mode 100644. The Azure deploy path invoked deploy.sh as "bash deploy.sh" so it never mattered, but the forced-command wrapper exec's it directly and got exit 126. deploy.sh also chmod +x's the scripts at deploy time, which cannot help the script that has to run first.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first SSH deploy failed twice, for two separate reasons. Both are fixed here.
1.
exit 126— Permission deniedAll three files in
scripts/are mode100644in git:The Azure path invoked the script as
bash deploy.sh, so the missing execute bit never mattered. The forced-command wrapperexecs it directly and gotexit 126. There's also a chicken-and-egg in the script itself — it runschmod +x scripts/*.sh, which cannot help the one script that has to run first.Fixed with
git update-index --chmod=+xon all three.2. The rerun executed the old script
After fixing the wrapper, the rerun got all the way through SSH, host-key verification and the git sync — then printed:
That text and that unconditional
docker loginwere removed in #4. The checkout had already been updated to the new commit moments earlier.Cause:
deploy.shreplaces its own file viagit reset --hard, and bash reads a script incrementally by byte offset. After the file is swapped, execution continues reading the new file from the old offset — so any deploy that changesdeploy.shruns a blend of both versions. It happened to land mid-way through the old logic here, but the failure mode is arbitrary.This has been latent since the Compose migration; the Azure path hit it too, it just never changed
deploy.shin a way that showed.Fix
Two stages:
Stage 1 only syncs, then re-execs. Stage 2 is what the updated copy runs.
COMMIT_SHAis exported across the re-exec and still takes precedence over the stale value in.env.Note
The host wrapper was also changed from
exec /opt/.../deploy.shtoexec bash /opt/.../deploy.sh, so a missing execute bit can never block a deploy again even if the index mode is lost.docs/adr/adr_02_ssh_deploy.mdis updated to match.