diff --git a/docs/usage/object-storage/amazon-s3.mdx b/docs/usage/object-storage/amazon-s3.mdx index 290be86..f3db377 100644 --- a/docs/usage/object-storage/amazon-s3.mdx +++ b/docs/usage/object-storage/amazon-s3.mdx @@ -175,6 +175,10 @@ By default, external access also uses the engine pod's own AWS identity. That id An intermediary role gives external access a stable identity instead. When you set one, the engine assumes the intermediary role first, and then assumes the external role from there, rather than using its own pod identity. Because the intermediary role ARN is stable and known ahead of time, you can share it with third parties and reference it in S3 bucket policies, IAM role trust policies, and AWS accounts outside your own organization. Access to the object storage bucket always uses the engine pod's own identity, so the intermediary role applies only to external locations. +Because a credential-less external location runs as the engine pod's identity, plan the identity's bucket access as the boundary. The engine rejects a credential-less URL that names the bucket in `managed_table_bucket_name`, so managed tablet data is not readable or writable that way. To use that bucket as an external location anyway, supply `CREDENTIALS` on the location: the query then authenticates as that principal rather than as the engine. Every other bucket the pod identity can reach stays reachable by URL, because an inline `s3://` URL carries no location privileges. + +The rejection does not apply when you set `default_s3_endpoint_override`, described under [Use an S3-compatible endpoint](#use-an-s3-compatible-endpoint). Managed storage and external locations then share one endpoint and one store, and reading the managed bucket by URL is part of that setup. + Create the intermediary IAM role and grant the engine's identity permission to assume it. The intermediary role's trust policy must allow the engine ServiceAccount identity to assume it, and the role needs only `sts:AssumeRole` on the external roles it is allowed to reach. Set the intermediary role ARN under `customEngineConfig.storage.aws.intermediary_access_role`: diff --git a/docs/usage/object-storage/azure-blob-storage.mdx b/docs/usage/object-storage/azure-blob-storage.mdx index 09eba7f..66d8ccb 100644 --- a/docs/usage/object-storage/azure-blob-storage.mdx +++ b/docs/usage/object-storage/azure-blob-storage.mdx @@ -184,6 +184,12 @@ By default, external access also uses the engine pod's own Azure identity. That An intermediary service principal gives external access a stable identity instead. When you set one, the engine uses the intermediary service principal for external access rather than its own pod identity. Because the service principal is stable and known ahead of time, you can share it with third parties and reference it in container role assignments, including on Azure subscriptions outside your own organization. Access to the object storage container always uses the engine pod's own identity, so the intermediary service principal applies only to external locations. +An `azure://` location carries no secret, because its `CREDENTIALS` clause supplies only a tenant ID. A query that reads or writes such a location therefore signs with an identity that this deployment owns: the engine pod's identity, or the intermediary service principal if you set one. What those identities can reach in Azure Blob Storage is the security boundary. + +The engine refuses an external location that points to the managed blob container itself, which is the container that `managed_table_bucket_name` names inside the storage account that `azure.storage_account_name` names. Managed tablet data is not readable or writable that way. + +Every other container those identities can reach stays reachable by URL. An inline `azure://` URL carries no location privileges, so any user who can run a query can read, overwrite, or delete the objects in such a container. Give the engine pod's identity blob access to the managed container only, and set an intermediary service principal for the external data. The pod identity then needs no access to that data itself. + Create the intermediary service principal and grant it the permissions it needs to reach the external data. Set its application client ID under `customEngineConfig.storage.azure.intermediary_service_principal_client_id`: diff --git a/docs/usage/object-storage/google-cloud-storage.mdx b/docs/usage/object-storage/google-cloud-storage.mdx index ec80c87..f0eb9c2 100644 --- a/docs/usage/object-storage/google-cloud-storage.mdx +++ b/docs/usage/object-storage/google-cloud-storage.mdx @@ -145,6 +145,24 @@ By default, external access also uses the engine pod's own Google identity. That An intermediary service account gives external access a stable identity instead. When you set one, the engine impersonates the intermediary service account for external access rather than using its own pod identity. Because the service account is stable and known ahead of time, you can share it with third parties and reference it in bucket IAM policies, including on Google Cloud projects outside your own organization. Access to the object storage bucket always uses the engine pod's own identity, so the intermediary service account applies only to external locations. +A `gs://` location that carries `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (a Google Cloud Storage HMAC key) is the exception: the query authenticates with that key through the S3-compatible XML API, bypassing both the pod identity and the intermediary service account. + +Where nothing provides Application Default Credentials to the engine pods, a credential-less `gs://` read is sent unauthenticated and reaches publicly readable objects only, and writing needs an identity and fails. That is the case on a cluster outside Google Cloud, and on GKE without Workload Identity and without `GOOGLE_APPLICATION_CREDENTIALS`. + +The pod identity is whatever Application Default Credentials resolve to inside the engine container. Google's libraries look for them in a fixed order, and the first match wins: + +1. `GOOGLE_APPLICATION_CREDENTIALS`, if it is set. The file it names is used as given, whether that is a mounted service account key or a credential configuration file for Workload Identity Federation. +2. `$HOME/.config/gcloud/application_default_credentials.json`, if that file exists. This is where `gcloud auth application-default login` writes, so it is normally present only on a workstation. +3. The instance metadata server, which is what Workload Identity Federation for GKE provides. + +Step 2 takes precedence over step 3, so an engine whose `HOME` holds a `gcloud` login authenticates as that person instead of as the pod identity. To pin the source, set `GOOGLE_APPLICATION_CREDENTIALS` on the engine container, or set `HOME` to a directory that only the engine writes to. The engine logs the source it resolved once per process, as `Using Google Cloud identity from `. + +You can also keep external access off the pod identity altogether, with `customEngineConfig.storage.gcp.allow_engine_identity: false`. A credential-less `gs://` read is then sent unauthenticated and reaches publicly readable objects only, and a write fails with an error that asks for credentials, so an external `gs://` location needs either an HMAC key or an intermediary service account. The chart renders `true` for the engines it installs, because the pod identity belongs to this chart release and is your own; the engine itself refuses that identity until a deployment permits it, because it cannot tell whose identity it runs as. When an intermediary service account is set, that account is the principal, so the setting has no effect. + +Because a credential-less external location runs as the engine pod's identity, plan the identity's bucket access as the boundary. The engine rejects a credential-less URL that names the bucket in `managed_table_bucket_name`, so managed tablet data is not readable or writable that way. To use that bucket as an external location anyway, supply `CREDENTIALS` on the location: the query then authenticates as that principal rather than as the engine. + +Every other bucket the pod identity can reach stays reachable by URL. An inline `gs://` URL carries no location privileges, so any user who can run a query can read, overwrite, or delete objects in those buckets. Grant the pod identity object access only on the managed bucket, and set an intermediary service account for external data so the pod identity itself needs no access to it. + Create the intermediary Google service account, grant the engine's identity `roles/iam.serviceAccountTokenCreator` on it, and grant the intermediary the permissions it needs to reach the external data. Set its ID under `customEngineConfig.storage.gcp.intermediary_service_account_id`: @@ -158,7 +176,7 @@ customEngineConfig: intermediary_service_account_id: projects/my-project/serviceAccounts/firebolt-intermediary@my-project.iam.gserviceaccount.com ``` -The chart passes the `storage.gcp` block through unchanged. The block is valid only when `managed_table_storage` is `gcs`. +The chart passes the `storage.gcp` block through unchanged, except for `allow_engine_identity`, which it renders as `true` unless you set it. The connection settings in the block apply to managed tables when `managed_table_storage` is `gcs`; `allow_engine_identity` governs external `gs://` locations, so it applies whatever backs managed tables. ## Storage scope diff --git a/helm/templates/_helpers.tpl b/helm/templates/_helpers.tpl index 67fe1f4..1ab0afb 100644 --- a/helm/templates/_helpers.tpl +++ b/helm/templates/_helpers.tpl @@ -118,6 +118,9 @@ canonical document has shape: metadata_endpoint: logging: format: json + storage: + gcp: + allow_engine_identity: true .Values.customEngineConfig is deep-merged on top of the canonical document at the root: keys at the top become siblings of `engine` @@ -137,6 +140,13 @@ before the merge: `schema_version`, `engine.id`, `engine.nodes`, `endpoints.http.listeners`. The same customEngineConfig therefore stays portable across chart versions. +`storage.gcp.allow_engine_identity` is a default rather than a +chart-authoritative path: the chart renders `true`, and +`customEngineConfig.storage.gcp.allow_engine_identity: false` takes it +back. It is read out of the user document before the merge rather than +overridden by it, because a boolean false is an empty value to +mergeOverwrite. + When `auth.enabled` is true, `instance.auth.{enabled,admin, local.signing_keys}` are built from `auth.admin` / `auth.signingKeys` using chart-owned secret mount paths (see engine-statefulset.yaml for @@ -250,11 +260,32 @@ Usage: {{ include "fbinstance.engineConfig" (dict "root" $ "engine" $engine) }} ) -}} {{- end -}} +{{/* + A credential-less external gs:// location authenticates as the engine pod's own + Google identity. The engine refuses that unless a deployment permits it, because + it cannot tell whose identity it runs as. An engine this chart installs runs under + a ServiceAccount belonging to whoever installed it, so the identity is theirs to + use, and they should not have to name the setting to reach their own buckets. + + Resolved from the user document here rather than left to the merge below: a + boolean false is an empty value to mergeOverwrite, so relying on the merge to let + a user turn this off would put a security answer at the mercy of merge semantics. +*/}} +{{- $allowEngineIdentity := true -}} +{{- if and (hasKey $user "storage") (kindIs "map" $user.storage) -}} +{{- if and (hasKey $user.storage "gcp") (kindIs "map" $user.storage.gcp) -}} +{{- if hasKey $user.storage.gcp "allow_engine_identity" -}} +{{- $allowEngineIdentity = $user.storage.gcp.allow_engine_identity -}} +{{- end -}} +{{- end -}} +{{- end -}} + {{- $canonical := dict "schema_version" "1.0" "engine" (dict "id" $engine.name "nodes" $nodes "termination_grace_period" (printf "%ds" $shutdownWait)) "instance" $instanceCanonical "logging" (dict "format" "json") + "storage" (dict "gcp" (dict "allow_engine_identity" $allowEngineIdentity)) -}} {{- if $root.Values.tls.engine.enabled -}} {{- $_ := set $canonical "endpoints" (dict "http" (dict "listeners" (list (dict