Skip to content

test: smoke-check the per-PR preview pipeline - #6205

Open
nicacioliveira wants to merge 2 commits into
mainfrom
test/preview-smoke
Open

test: smoke-check the per-PR preview pipeline#6205
nicacioliveira wants to merge 2 commits into
mainfrom
test/preview-smoke

Conversation

@nicacioliveira

@nicacioliveira nicacioliveira commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

First end-to-end run of per-PR preview environments, landed earlier today in #6125 and decocms/deco-apps-cd#244.

Do not merge. The only change is a comment in deploy/preview/README.md; this PR exists to exercise the pipeline and will be closed once verified.

What this is checking

Stage Expected
preview-build.yaml two images tagged pr-<n>-<7 of head sha>, sticky comment with the URL
ApplicationSet generator picks the PR up within ~60s of the label
Sync waves ExternalSecret (-30) → Postgres + MinIO (-20) → migrate Job (-10) → app (0)
Pods six, all on the studio-preview spot node pool
Storage DATABASE_URL and S3_ENDPOINT pointing at the in-namespace Postgres and MinIO
Routing HTTPRoute attaches, attachedRoutes on the shared Gateway goes 0 → 1
Teardown closing this PR removes the namespace, and the database and bucket with it

Known gaps at this point

Nothing gates the preview URL yet — no oauth2-proxy or AuthorizationPolicy, and the record is DNS-only, so Cloudflare is not in front either. Fine for a short-lived smoke test that gets closed; not fine for sharing the link around.

Hosted agent sandboxes, Google sign-in, billing, monitoring and outbound email are all out of scope for a preview by design — see deploy/preview/README.md.


Summary by cubic

Smoke-tests the per-PR preview pipeline by adding a no-op comment in deploy/preview/README.md. Also rebuilds preview images as multi-arch to prevent failures when pods reschedule onto ARM (Graviton) nodes. Do not merge; close after verification.

  • Builds two multi-arch images tagged pr-<n>-<7-of-head-sha> and posts a sticky preview URL comment.
  • ApplicationSet picks the labeled PR within ~60s; sync order: ExternalSecret → Postgres/MinIO → migrate Job → app.
  • Runs six pods on the studio-preview spot pool; DATABASE_URL and S3_ENDPOINT point to in-namespace Postgres/MinIO.
  • HTTPRoute attaches to the shared Gateway; closing the PR deletes the namespace, database, and bucket.

Caveats

  • No auth on the preview URL; DNS-only routing (no Cloudflare).
  • Hosted sandboxes, Google sign-in, billing, monitoring, and outbound email are out of scope.

Written for commit 25d2d1c. Summary will update on new commits.

Review in cubic

@nicacioliveira nicacioliveira added the preview Deploy a per-PR preview environment label Aug 18, 2026
@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

🔍 Preview: https://

Built from pr-6205-398ba0c. Argo CD picks the image up
within ~60s; the first sync also creates and migrates the database, so
allow another minute or two on a brand-new preview.

Sign in: the database is empty and yours — sign up with any email
and password. It is thrown away when this PR closes.

What does not work in a preview
  • Agent tool execution against a hosted sandbox — previews run
    STUDIO_SANDBOX_PROVIDER=user-desktop with no daemon attached, so
    dispatch returns 409 link_offline. Sandbox previews and the
    sandbox lifecycle UI are equally out.
  • AI features until you add your own provider key in org settings.
    Previews ship no key, so preview LLM spend is zero by construction.
  • Google / GitHub sign-in — OAuth callbacks cannot be registered
    for a per-PR hostname. The buttons are hidden.
  • Monitoring dashboard (no ClickHouse), billing (no Stripe),
    outbound email (no mail provider).
  • Multi-pod behaviour — a preview is one pod. Do not conclude
    "it worked in preview" about a distributed-systems change; that is
    what tests/multi-pod/ is for.

Remove the preview label to tear this down now. Previews expire
48h after their last deploy — push, or re-add the label, to
reset the clock.

nicacioliveira added a commit that referenced this pull request Aug 18, 2026
…ts it (#6208)

The migrate Job takes both the ConfigMap and the Secret via envFrom. The
ExternalSecret was placed at wave -30 for exactly that reason, and its comment
says so — but the ConfigMap got no wave at all, so it landed at the default 0,
after the Job at -10.

The Job is then scheduled with a configMapRef that does not exist yet, sits in
CreateContainerConfigError, and the whole sync stalls waiting for the hook to
complete. Nothing on any object reports a problem; the Application just says
OutOfSync/Missing with 'waiting for completion of hook'.

Found on the very first real preview (#6205). helm template renders but never
sequences, so no amount of render assertions could have caught it — helm-test
now walks every Job, Deployment and StatefulSet, resolves each envFrom source,
and fails when a consumer is sequenced before something it mounts. Verified
against the pre-fix chart: it flags the exact pair.

Fixes the chart-managed Secret too, which had the same gap for non-ESO installs.
@nicacioliveira nicacioliveira removed the preview Deploy a per-PR preview environment label Aug 19, 2026
nicacioliveira added a commit that referenced this pull request Aug 19, 2026
Argo's CreateNamespace=true creates a namespace without adopting it: no tracking
annotation, never part of the Application's resource tree. So a cascading delete
removed everything inside — pods, Services, HTTPRoute, ConfigMaps, and the
Postgres and MinIO holding the data — and left an empty namespace behind. One
per pull request that ever had a preview, accumulating quietly.

Observed removing the label from #6205: every resource gone, attachedRoutes back
to 0, the URL 404, and studio-pr-6205 still Active with no ownerReferences and no
argocd tracking-id.

The chart now renders the Namespace at wave -40, so it is owned like everything
else and deleted with them. Three README files promised this behaviour already;
it is now true.

It also carries the Gateway's selector label, which removes a second failure:
route attachment no longer depends on whoever creates the Application setting
managedNamespaceMetadata correctly — the omission the troubleshooting guide
calls the most likely cause of a silent 404.

Chart 0.14.2 -> 0.14.3.
Trivial doc comment. Exists only to exercise the preview build, the
ApplicationSet generator and the teardown path end to end for the first time.
Close without merging once the preview has been verified.
The images for this PR predate #6211 and are amd64 only, so the preview died
when a spot reclaim moved it onto a Graviton node.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant