fix(preview): the ServiceAccount is created after the Job that runs as it - #6223
Merged
Conversation
…s it Second instance of the same mistake, one identifier further out. The ConfigMap got a sync-wave at 0.14.1 because the migration Job mounts it via envFrom; the ServiceAccount the Job *runs as* never got one, so it landed at the default 0 — after the Job at -10. This fails more quietly than the ConfigMap did. The Job gets no pod at all: Error creating: pods "studio-pr-6205-preview-migrate-" is forbidden: error looking up service account studio-pr-6205/studio-pr-6205: not found So there is no pod to inspect, no container logs, no CreateContainerConfigError — just a Job at Running 0/1 and an Application stalled on the hook. Postgres and MinIO come up fine and the namespace looks half-alive indefinitely. Found on the second end-to-end run of #6205, after the ConfigMap fix had already shipped. The assertion added for the ConfigMap only looked at envFrom, which is why it did not catch this. It now walks everything a pod names and the kubelet must resolve before it can start — serviceAccountName, envFrom sources, and volume configMap/secret references — and credits an ExternalSecret's wave to the Secret it produces. Verified against main: it flags this exact pair.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second run of #6205 end to end. It stalled in the same place as yesterday, for a different reason.
The same mistake, one identifier further out
0.14.1 gave the
-configConfigMap a sync-wave because the migration Job mounts it viaenvFrom. The ServiceAccount the Job runs as never got one, so it landed at the default0— after the Job at-10.It fails more quietly than the ConfigMap did
With a missing ConfigMap the pod at least exists and reports
CreateContainerConfigError. Here the kubelet refuses to create the pod at all, so there is nothing to inspect: no pod, no container logs, no events on anything but the Job. JustRunning 0/1and an Application stuck onwaiting for completion of hook.Postgres and MinIO come up fine, so the namespace looks half-alive indefinitely.
Why the existing assertion missed it
The check added alongside the ConfigMap fix only walked
envFromsources.serviceAccountNameis not anenvFrom.It now covers everything a pod names that the kubelet must resolve before it can start:
serviceAccountNameenvFrom→configMapRef/secretRefconfigMap/secretand credits an
ExternalSecret's wave to theSecretit produces, since the Secret does not exist as a rendered object.Verified against
main: it flags the exact pair.12 references checked on the fixed chart, all in order.
Note on how this class of bug behaves
Neither instance is visible to
helm template— rendering never sequences. Both were found by a real sync, and both looked like something else at first glance. That is the argument for the assertion covering the whole class rather than the two cases we happen to have hit.Verification
main.helm lintclean.preview.enabled, so nothing outside previews changes.Chart
0.14.4→0.14.5.Summary by cubic
Ensure the ServiceAccount is created before the preview migration Job that runs as it. Previously the ServiceAccount had no sync-wave (0) and applied after the Job (-10), so the kubelet refused to create the pod and the sync stalled.
-30to the ServiceAccount.helm-testto coverserviceAccountName,envFromconfigMap/secret, volume configMap/secret, and credit anExternalSecret’s wave to itsSecret..Values.preview.enabled; no effect outside previews. Bump chart to 0.14.5.Written for commit 6e87210. Summary will update on new commits.