Skip to content

fix(preview): the ServiceAccount is created after the Job that runs as it - #6223

Merged
nicacioliveira merged 1 commit into
mainfrom
fix/preview-sa-sync-wave
Aug 19, 2026
Merged

fix(preview): the ServiceAccount is created after the Job that runs as it#6223
nicacioliveira merged 1 commit into
mainfrom
fix/preview-sa-sync-wave

Conversation

@nicacioliveira

@nicacioliveira nicacioliveira commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Second run of #6205 end to end. It stalled in the same place as yesterday, for a different reason.

Error creating: pods "studio-pr-6205-preview-migrate-" is forbidden:
  error looking up service account studio-pr-6205/studio-pr-6205: not found

The same mistake, one identifier further out

0.14.1 gave the -config ConfigMap a sync-wave because the migration Job mounts it via envFrom. The ServiceAccount the Job runs as never got one, so it landed at the default 0 — after the Job at -10.

resource wave
Namespace -40
ConfigMap, ExternalSecret -30
Postgres, MinIO -20
migrate Job -10
ServiceAccount 0 ← the Job runs as it

It fails more quietly than the ConfigMap did

With a missing ConfigMap the pod at least exists and reports CreateContainerConfigError. Here the kubelet refuses to create the pod at all, so there is nothing to inspect: no pod, no container logs, no events on anything but the Job. Just Running 0/1 and an Application stuck on waiting for completion of hook.

Postgres and MinIO come up fine, so the namespace looks half-alive indefinitely.

Why the existing assertion missed it

The check added alongside the ConfigMap fix only walked envFrom sources. serviceAccountName is not an envFrom.

It now covers everything a pod names that the kubelet must resolve before it can start:

  • serviceAccountName
  • envFromconfigMapRef / secretRef
  • volumes → configMap / secret

and credits an ExternalSecret's wave to the Secret it produces, since the Secret does not exist as a rendered object.

Verified against main: it flags the exact pair.

::error::a resource is sequenced before something it needs:
  Job/studio-pr-42-preview-migrate (wave -10) needs ServiceAccount/studio-pr-42 (wave 0)

12 references checked on the fixed chart, all in order.

Note on how this class of bug behaves

Neither instance is visible to helm template — rendering never sequences. Both were found by a real sync, and both looked like something else at first glance. That is the argument for the assertion covering the whole class rather than the two cases we happen to have hit.

Verification

  • Default render outside preview: 0 lines of difference against main.
  • helm lint clean.
  • The wave is gated on preview.enabled, so nothing outside previews changes.

Chart 0.14.40.14.5.


Summary by cubic

Ensure the ServiceAccount is created before the preview migration Job that runs as it. Previously the ServiceAccount had no sync-wave (0) and applied after the Job (-10), so the kubelet refused to create the pod and the sync stalled.

  • Add preview-only sync-wave -30 to the ServiceAccount.
  • Extend the CI ordering check in helm-test to cover serviceAccountName, envFrom configMap/secret, volume configMap/secret, and credit an ExternalSecret’s wave to its Secret.
  • Gate changes behind .Values.preview.enabled; no effect outside previews. Bump chart to 0.14.5.

Written for commit 6e87210. Summary will update on new commits.

Review in cubic

…s it

Second instance of the same mistake, one identifier further out. The ConfigMap
got a sync-wave at 0.14.1 because the migration Job mounts it via envFrom; the
ServiceAccount the Job *runs as* never got one, so it landed at the default 0 —
after the Job at -10.

This fails more quietly than the ConfigMap did. The Job gets no pod at all:

  Error creating: pods "studio-pr-6205-preview-migrate-" is forbidden:
  error looking up service account studio-pr-6205/studio-pr-6205: not found

So there is no pod to inspect, no container logs, no CreateContainerConfigError
— just a Job at Running 0/1 and an Application stalled on the hook. Postgres and
MinIO come up fine and the namespace looks half-alive indefinitely.

Found on the second end-to-end run of #6205, after the ConfigMap fix had already
shipped.

The assertion added for the ConfigMap only looked at envFrom, which is why it
did not catch this. It now walks everything a pod names and the kubelet must
resolve before it can start — serviceAccountName, envFrom sources, and volume
configMap/secret references — and credits an ExternalSecret's wave to the Secret
it produces. Verified against main: it flags this exact pair.
@nicacioliveira
nicacioliveira merged commit 9decac7 into main Aug 19, 2026
32 checks passed
@nicacioliveira
nicacioliveira deleted the fix/preview-sa-sync-wave branch August 19, 2026 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant