feat(secrets): gsm:// / env:// reference resolution in config (Google Secret Manager) - #80
Merged
Merged
Conversation
…ecret Manager)
Add scheme-dispatched reference resolution so secrets stay out of YAML. A config
string can reference a managed secret via a URI scheme, resolved once over the raw
config before Pydantic validation — connectors need no changes.
- gsm://<id> (latest), gsm://<id>/versions/<N> (pinned), or full resource path;
Google Secret Manager via Application Default Credentials
- env://<VAR> built-in scheme (works inline, unlike legacy whole-value BIZON_ENV_)
- inline ${...} tokens inside larger strings (e.g. DSNs), multiple per value
- optional top-level secrets: block for provider defaults (secrets.gsm.project_id)
- bizon secrets check <config> to dry-run/validate references (masked output)
- new optional extra: bizon[secretmanager]
- legacy BIZON_ENV_ references keep working unchanged
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Keep secrets out of pipeline YAML by referencing them with a URI scheme instead of plaintext. Resolution runs once over the raw config dict before Pydantic validation, so connectors keep reading plain strings and need no changes — same mechanism as the existing
BIZON_ENV_feature, generalized.Google Secret Manager is the first provider; the design is factory-extensible (
awssm://,vault://later = one adapter + one registry entry).Reference syntax
token: gsm://notion-api-tokenkey: gsm://bq-sa-key/versions/3gsm://projects/<p>/secrets/<id>/versions/<n>dsn: "postgres://u:${gsm://db-pw}@host/db"env://MY_TOKEN(also works inline)gsm://authenticates via Application Default Credentials (workload identity / ambient creds).secrets:block holds provider defaults (e.g.secrets.gsm.project_id).postgresql://...,https://...) are never mistaken for references (unknown scheme → left untouched).BIZON_ENV_FOOkeeps working unchanged.CLI
bizon secrets check <config>dry-runs every reference and reports resolved/failed with masked output (length only), exiting non-zero on failure — catch typos / missing secrets / ADC issues before a real run.Changes
bizon/engine/resolvers/—AbstractReferenceResolver,ResolverRegistry(lazy per-scheme build + per-run cache),resolve_references_in_config,collect_references_in_config, andenv/gcp.gsmadapters.bizon/engine/engine.py— resolver pass after the legacy env pass.bizon/common/models.py— optionalsecretsfield onBizonConfig.bizon/cli/main.py—bizon secrets check.pyproject.toml— new optional extrasecretmanager(google-cloud-secret-manager); imported lazily.Testing
tests/engine/test_reference_resolution.py) covering whole-value/inline/pinned/full-pathgsm,env, caching, list fields, error cases, andGSMResolverpath-building (mocked GSM client — no real GCP). All pass.tests/enginesuite: 65 passed; the only failures are pre-existingmy_pg_backendtests that require a live Postgres (unrelated).ruff format+ruff checkclean.🤖 Generated with Claude Code