Skip to content

fix(deploy): the setup hook cannot authenticate on an HTTPS deployment (CHOO-2067) - #289

Open
christian-mcdermott wants to merge 1 commit into
mainfrom
fix/setup-hook-secure-cookie
Open

fix(deploy): the setup hook cannot authenticate on an HTTPS deployment (CHOO-2067)#289
christian-mcdermott wants to merge 1 commit into
mainfrom
fix/setup-hook-secure-cookie

Conversation

@christian-mcdermott

Copy link
Copy Markdown
Collaborator

The post-upgrade setup hook fails on every deployment served over HTTPS, and has been worked around with --no-hooks rather than diagnosed. Found while deploying switch-dev, where revision 71 failed on it and the two revisions after it skipped it.

Why it fails

Such a deployment sets GATEWAY_COOKIE_SECURE, correctly, so the gateway marks the switch_auth session cookie Secure. The setup job runs inside the cluster and talks to http://<service>:8000.

The interesting part is that a cookie jar accepts a Secure cookie over http — it just never sends one back on a non-https request. So the login looks perfect: 200, cookie visibly in the jar. Every authorized call after it goes out bare and comes back 401, with nothing naming the cause. That asymmetry is why this presented as an auth failure deep in the script rather than as a login error, and it has its own test.

The fix re-sets the cookie without the flag. Same token; only the rule about which schemes it may travel on is set aside, on a connection that never leaves the cluster.

The second defect, which the first one exposed

resp = client.get("/gateway/collaborations")
if resp.is_success:
    ...  # return the existing bridge
# falls through
client.post("/gateway/collaborations", json={..., "set_as_default": True})

Any failure on that read — a 401, a timeout, a 503 — reads as "there are no bridges", and the script goes on to register a second Mattermost bridge and make it the default, taking that from whatever held it. On switch-dev it only stopped because the create was refused as well. Had the read alone failed, the deployment would have quietly gained a duplicate bridge and a new default.

A read that fails is not an empty list. It raises now, and the caller can retry — which a duplicated bridge cannot be undone by.

Testing

Seven tests, including one that pins the cookie behaviour itself so the premise can't silently stop being true, and one that asserts a 401 on the read posts nothing.

1775 passed on the full suite; lint and types clean.

Note

Reviewed against main. The switch-dev deployment currently runs --no-hooks; once this lands, the hook can be left on again.

🤖 Generated with Claude Code

The post-upgrade hook has been failing on every deployment served over
HTTPS, and being worked around with --no-hooks rather than read.

Such a deployment sets GATEWAY_COOKIE_SECURE, so the gateway marks the
switch_auth session cookie Secure. The setup job runs inside the cluster
against plain http://, and a cookie jar will accept a Secure cookie there
but refuse to send one back. The login therefore looks perfect — 200, and
the cookie visibly stored — while every authorized call after it goes out
with no session and comes back 401, naming nothing. Re-set the cookie
without the flag: same token, and the connection never leaves the
cluster.

The second half is what that 401 then did. register_bridge asked "is a
bridge already registered?" with `if resp.is_success:` and treated any
failure as "there are none", falling through to register a SECOND
Mattermost bridge with set_as_default — taking the default from whatever
held it. Here it happened to die because the create was refused too; had
only the read been refused, a deployment would have quietly acquired a
duplicate bridge and a new default. A read that fails is not an empty
list, so it raises now.

Tests cover both, and pin the cookie behaviour itself — accepted over
http, withheld on the way back — because that asymmetry is the whole
reason this presented as a 401 rather than as a login error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant