feat(helm): let engines use their pod Google identity for gs:// (FB-2939)

[simon-firebolt]

Background

FB-2939: a credential-less external gs:// location authenticates as the engine
pod's own Google identity. The engine refuses to do that unless a deployment
permits it through storage.gcp.allow_engine_identity, because the engine cannot
tell whose identity it runs as: on a shared service that identity belongs to the
service and not to the caller of the query, so the engine's own default is off.

The engine-side change is firebolt-analytics/packdb#24760.

Summary

An engine this chart installs runs under a ServiceAccount belonging to whoever
installed it, so the identity is theirs, and they should not have to name a
setting to reach their own buckets. This is the layer that knows a deployment is
self-managed, so this is where the answer belongs.

• The canonical document in fbinstance.engineConfig carries
  storage.gcp.allow_engine_identity: true, and
  customEngineConfig.storage.gcp.allow_engine_identity: false takes it back.
• The value is read out of the user document before the merge rather than
  overridden by it. A boolean false is an empty value to mergeOverwrite, so
  leaving the override to the merge would rest a security answer on merge
  semantics. Reading it first makes the merge a no-op for this key either way,
  whichever way mergo treats a zero value.
• allow_engine_identity is documented as a chart default, not a
  chart-authoritative path: it is not stripped from user input, unlike
  schema_version / engine.id / instance.type and the rest.
• The object-storage pages pick up the rest of this work: which principal signs
  an external request per cloud, where the pod's Application Default Credentials
  come from and in what order they are tried, and that the managed bucket is
  rejected as an external location.

Draft until the engine image carries the key. An engine whose config parser
predates packdb#24760 rejects an unknown configuration entry outright:
ChangeParser reports "Configuration entry does not exist." and Load() throws,
which aborts startup. So this must not merge before appVersion pins an engine
image that includes the packdb change. Today it pins
release-5.0.0-pre.0.20260815074041.524de236b514.

Test Plan

• Not run locally: no helm binary on the authoring machine, so make lint
  (helm lint --strict plus helm template) has not been executed. CI is the
  first real check on this diff.
• Worth confirming in review by rendering both arms, since the point of reading
  the value before the merge is that neither depends on mergo's zero-value
  behaviour:
  helm template r helm shows storage.gcp.allow_engine_identity: true, and
  helm template r helm --set customEngineConfig.storage.gcp.allow_engine_identity=false
  shows false.

---

Note

Medium Risk
Changes a security-sensitive default so credential-less gs:// queries use the engine pod identity. Wrong IAM on that identity, or shipping this config before the engine parser knows the key, can widen data access or fail engine startup.

Overview
The chart now opts engines into using the pod’s Google identity for credential-less external gs:// locations. Canonical engine config includes storage.gcp.allow_engine_identity: true; operators can still set it to false in customEngineConfig.

The flag is read from user values before mergeOverwrite, so a boolean false is not dropped as an empty merge value. It is a default, not a chart-authoritative path.

Object-storage docs spell out the identity boundary: managed-bucket URLs are rejected without CREDENTIALS, inline URLs inherit whatever the pod (or intermediary) can reach, GCS ADC resolution order, HMAC bypass, and the S3-compatible exception for default_s3_endpoint_override.

^{Reviewed by Cursor Bugbot for commit e031a1c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[fireboltdb-docs-integration]

Mintlify docs preview

Docs were aggregated into the Firebolt documentation site.

Status	Preview	Updated (UTC)
🟢 Ready	View Preview	Aug 21, 2026, 1:26 PM

• Packdb PR (draft): https://github.com/firebolt-analytics/packdb/pull/25025
• Aggregate branch: aggregate/helm-docs-pr-79

Mintlify may take a few minutes to finish building. This comment updates on each sync while the source PR is open.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit e031a1c. Configure here.}

[cursor]

Config key breaks engine startup

High Severity

fbinstance.engineConfig always writes storage.gcp.allow_engine_identity into every engine config.yaml, including S3/Azure and default installs. Chart.yaml appVersion still pins an image that predates packdb#24760, so ChangeParser rejects the unknown entry and Load() aborts startup for all engines on upgrade.

 

^{Reviewed by Cursor Bugbot for commit e031a1c. Configure here.}