fix(preview): the config ConfigMap is created after the Job that mounts it - #6208
Merged
Conversation
…ts it The migrate Job takes both the ConfigMap and the Secret via envFrom. The ExternalSecret was placed at wave -30 for exactly that reason, and its comment says so — but the ConfigMap got no wave at all, so it landed at the default 0, after the Job at -10. The Job is then scheduled with a configMapRef that does not exist yet, sits in CreateContainerConfigError, and the whole sync stalls waiting for the hook to complete. Nothing on any object reports a problem; the Application just says OutOfSync/Missing with 'waiting for completion of hook'. Found on the very first real preview (#6205). helm template renders but never sequences, so no amount of render assertions could have caught it — helm-test now walks every Job, Deployment and StatefulSet, resolves each envFrom source, and fails when a consumer is sequenced before something it mounts. Verified against the pre-fix chart: it flags the exact pair. Fixes the chart-managed Secret too, which had the same gap for non-ESO installs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found on the first real preview, #6205. The environment never came up.
What happens
The migrate Job takes both the ConfigMap and the Secret via
envFrom. TheExternalSecretsits at wave-30for exactly that reason, and its own comment says so — "ahead of everything that mounts it: the migrate Job (-10) and the app Deployments (0) both consume this Secret via envFrom". The ConfigMap in the sameenvFromhad no wave at all, so it landed at the default0.-configConfigMapThe Job is scheduled, the kubelet cannot resolve
configMapRef, and the pod parks inCreateContainerConfigError. Because the Job is a sync hook, the entire Application stalls on it:Nothing on any object reports a fault. Postgres and MinIO come up fine and stay
Running, so the namespace looks half-healthy indefinitely.Why no test caught it
helm templaterenders; it never sequences. Every existing assertion checks what the manifests say — that waves are orderedpostgres < migrate < app, that the sixS3_*keys are present, that pods carry the node-pool selector. None of them could observe that a consumer is scheduled before something it mounts, because that only exists at sync time.So this adds the assertion that generalises it: walk every Job, Deployment and StatefulSet, resolve each
envFromsource rendered by this chart, and fail when the source has a higher wave than its consumer. Verified against the pre-fix chart — it flags exactly the offending pair:and passes on the fixed one, checking 8 references.
Also
The chart-managed
Secret(rendered whenexternalSecret.enabled=false) had the same gap, so non-ESO installs would hit this too. Fixed alongside.Verification
helm templateof the default render againstmain: 0 lines of difference. The wave annotation is gated onpreview.enabled.helm lintclean.Chart
0.14.0→0.14.1.Rollout
Merging publishes
0.14.1. The previews chart indecocms/deco-apps-cdpinsstudioChart.version, so that pin needs bumping before any preview picks the fix up — #6205 is stalled until then.Summary by cubic
Prevents preview syncs from stalling by sequencing the config ConfigMap (and chart-managed Secret) before the migrate Job. Previously, the Job ran at wave -10 while the ConfigMap defaulted to 0, so pods hit CreateContainerConfigError; now both resources get wave -30 when
preview.enabled, ensuring they exist before the Job (-10) and app Deployments (0). Adds a CI check that fails if any workload is sequenced before anenvFromsource.preview.enabled, so the default render is unchanged. Required action: bumpstudioChart.versionindecocms/deco-apps-cdto0.14.1for previews to pick up the fix.Written for commit adfc1ac. Summary will update on new commits.