Skip to content

fix(preview): the config ConfigMap is created after the Job that mounts it - #6208

Merged
nicacioliveira merged 1 commit into
mainfrom
fix/preview-configmap-sync-wave
Aug 18, 2026
Merged

fix(preview): the config ConfigMap is created after the Job that mounts it#6208
nicacioliveira merged 1 commit into
mainfrom
fix/preview-configmap-sync-wave

Conversation

@nicacioliveira

@nicacioliveira nicacioliveira commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Found on the first real preview, #6205. The environment never came up.

What happens

The migrate Job takes both the ConfigMap and the Secret via envFrom. The ExternalSecret sits at wave -30 for exactly that reason, and its own comment says so — "ahead of everything that mounts it: the migrate Job (-10) and the app Deployments (0) both consume this Secret via envFrom". The ConfigMap in the same envFrom had no wave at all, so it landed at the default 0.

resource wave
ExternalSecret -30
Postgres, MinIO -20
migrate Job -10
-config ConfigMap 0 ← after the Job that mounts it

The Job is scheduled, the kubelet cannot resolve configMapRef, and the pod parks in CreateContainerConfigError. Because the Job is a sync hook, the entire Application stalls on it:

sync=OutOfSync health=Missing
phase=Running message=waiting for completion of hook batch/Job/studio-pr-6205-preview-migrate

Error: configmap "studio-pr-6205-config" not found

Nothing on any object reports a fault. Postgres and MinIO come up fine and stay Running, so the namespace looks half-healthy indefinitely.

Why no test caught it

helm template renders; it never sequences. Every existing assertion checks what the manifests say — that waves are ordered postgres < migrate < app, that the six S3_* keys are present, that pods carry the node-pool selector. None of them could observe that a consumer is scheduled before something it mounts, because that only exists at sync time.

So this adds the assertion that generalises it: walk every Job, Deployment and StatefulSet, resolve each envFrom source rendered by this chart, and fail when the source has a higher wave than its consumer. Verified against the pre-fix chart — it flags exactly the offending pair:

::error::a resource is sequenced before something it mounts:
  Job/studio-pr-42-preview-migrate (wave -10) consumes studio-pr-42-config (wave 0)

and passes on the fixed one, checking 8 references.

Also

The chart-managed Secret (rendered when externalSecret.enabled=false) had the same gap, so non-ESO installs would hit this too. Fixed alongside.

Verification

  • helm template of the default render against main: 0 lines of difference. The wave annotation is gated on preview.enabled.
  • helm lint clean.
  • The new assertion fails on the old chart and passes on the new one.

Chart 0.14.00.14.1.

Rollout

Merging publishes 0.14.1. The previews chart in decocms/deco-apps-cd pins studioChart.version, so that pin needs bumping before any preview picks the fix up — #6205 is stalled until then.


Summary by cubic

Prevents preview syncs from stalling by sequencing the config ConfigMap (and chart-managed Secret) before the migrate Job. Previously, the Job ran at wave -10 while the ConfigMap defaulted to 0, so pods hit CreateContainerConfigError; now both resources get wave -30 when preview.enabled, ensuring they exist before the Job (-10) and app Deployments (0). Adds a CI check that fails if any workload is sequenced before an envFrom source.

  • Chart v0.14.1; the wave annotation is gated on preview.enabled, so the default render is unchanged. Required action: bump studioChart.version in decocms/deco-apps-cd to 0.14.1 for previews to pick up the fix.

Written for commit adfc1ac. Summary will update on new commits.

Review in cubic

…ts it

The migrate Job takes both the ConfigMap and the Secret via envFrom. The
ExternalSecret was placed at wave -30 for exactly that reason, and its comment
says so — but the ConfigMap got no wave at all, so it landed at the default 0,
after the Job at -10.

The Job is then scheduled with a configMapRef that does not exist yet, sits in
CreateContainerConfigError, and the whole sync stalls waiting for the hook to
complete. Nothing on any object reports a problem; the Application just says
OutOfSync/Missing with 'waiting for completion of hook'.

Found on the very first real preview (#6205). helm template renders but never
sequences, so no amount of render assertions could have caught it — helm-test
now walks every Job, Deployment and StatefulSet, resolves each envFrom source,
and fails when a consumer is sequenced before something it mounts. Verified
against the pre-fix chart: it flags the exact pair.

Fixes the chart-managed Secret too, which had the same gap for non-ESO installs.
@nicacioliveira
nicacioliveira merged commit ae3686c into main Aug 18, 2026
30 checks passed
@nicacioliveira
nicacioliveira deleted the fix/preview-configmap-sync-wave branch August 18, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant