Skip to content

feat(lambda): external invoke permissions, function DLQ and scope identity metadata - #46

Open
andres-nullplatform wants to merge 22 commits into
mainfrom
feat/lambda-external-triggers
Open

feat(lambda): external invoke permissions, function DLQ and scope identity metadata#46
andres-nullplatform wants to merge 22 commits into
mainfrom
feat/lambda-external-triggers

Conversation

@andres-nullplatform

Copy link
Copy Markdown
Contributor

Covers the Spin gaps that live in this repo (CLIEN-1002). The scope covered the function well but not its invocation: Spin's lambdas are triggered by DynamoDB streams, SQS, API Gateway authorizers, EventBridge and S3, none of which the scope could reach.

What's in

Gap Change
External invoke permissions (blocking) triggers.invoke_permissions in the scope-configuration, reconciled onto the main alias
Function dead letter queue dead_letter_target_arn attribute, with an IAM grant scoped to that ARN
Scope identity lambda.function_name / function_arn / alias_arn / main_alias / execution_role_* published to the NRN
provided.al2 runtime added to the spec enum (kafka-dlt-manager uses it)

Out of scope: the SQS event source mapping and the missing stream ESM parameters (both in the SQS / DynamoDB services — the NRN identity above is their prerequisite), the EventBridge rules themselves (they stay in Spin's Terraform and only need the permission), and gaps 3/4 (VPC overrides, advanced logging), dropped by decision.

Why these run outside Terraform

The scope's tofu run only happens on create/delete. Its state holds the placeholder function while deployments mutate the real one through the AWS CLI, so an apply on update sees drift and reverts the function to the placeholder — publishing a placeholder version and moving the main alias to it. lifecycle.ignore_changes on aws_lambda_function.main is currently an empty block, and compute/lambda/setup also forces runtime=nodejs22.x on placeholder-zip deploys, which would break a .NET function regardless.

So invoke permissions and the DLQ are reconciled by idempotent AWS CLI scripts that run on both create and update and touch only the field they own. Making update-scope tofu-driven is real work with its own e2e validation — see Follow-ups.

Two safety properties worth reviewing closely:

  • Permissions target the alias, so external triggers follow blue/green weights.
  • Only Sids carrying the np-ext- prefix are ever removed. AllowAPIGatewayInvoke, AllowALBInvoke and hand-made statements are untouched (covered by tests).

Incidental fixes

  • update-scope was unreachable. It declares "type": "custom", map_custom_action had no case for it, so it looked for a non-existent update-scope.yaml.
  • delete-scope would have failed once a DLQ existed: IAM refuses DeleteRole while unmanaged inline policies remain. cleanup_role_inline_policies runs before the destroy.
  • Failure assertions in the bats suite were vacuous. These scripts are sourced by the engine and use return; run as bash <script>, return is a warning that doesn't stop execution, so the script always exited 0. wait_provisioned_concurrency looped forever for the same reason — its timeout branch couldn't return out of the poll loop. Locally this was masked by utils/log using ${level,,}, which is a fatal error on bash 3.2 and aborted scripts for the wrong reason.

Verification

ok / not ok, per suite, this branch vs main:

Suite main branch
scope 58 / 29 96 / 26
deployment (excl. polling) — / 76 — / 64
wait_provisioned_concurrency 6 / 12 11 / 7
new test files 39 / 0

No new failures anywhere; 15 pre-existing ones fixed. shellcheck clean on everything new, tofu fmt clean on the files touched.

Not yet verified end to end against a real AWS account — that's the check that matters for "does not alter code, env vars or the active version of a running lambda". Steps: create a scope with permissions + DLQ, deploy real code, run update-scope changing only the permissions, then confirm CodeSha256, Environment and the alias version are unchanged.

Follow-ups (not in this PR)

  • update-scope can't apply function configuration (memory, timeout, runtime, layers…); those still land on the next deployment.
  • ~97 pre-existing test failures remain: orphaned files (deployment/tests/scripts/store_nrn_metadata.bats points at a script that moved) and tests that shell out to the real AWS CLI.
  • The README "Scope Structure" tree still describes the old deployment/-rooted tofu layout.

map_custom_action had no case for it, so the workflow was unreachable.
These scripts use return, so `bash <script>` never propagates their exit
code and every assert_failure passed vacuously. Adds run_sourced to the
shared helper, and lowercases log levels with tr since ${level,,} aborts
on bash 3.2.
Reconciled by AWS CLI scripts on create and update, not by tofu: the
scope state holds the placeholder function, so an apply on update would
roll back deployed code. delete-scope drops the DLQ inline policy first,
or DeleteRole fails. Also adds the provided.al2 runtime.
Lets a service attach an event source mapping to the alias instead of
re-deriving names. update-scope backfills pre-existing scopes from AWS.
Also fixes a stale capability table and test paths.
Compare .Principal.AWS too, or an account principal churns remove+add on
every run. Fail on a get-policy error instead of treating it as an empty
policy. Restore the previous statement when a refresh add fails.
A dedicated execution role can be shared between scopes, so an unscoped
policy name let one scope overwrite or delete another's grant. The
discarded waiter status also reported success on a failed update.
jq --argjson fails on a non-numeric value, blanking the whole payload
while still reporting success.
The make targets are at the root, not under testing/.
providers.<category> is the provider's attributes object and triggers is a
top-level group, so the extra .lambda level resolved to null and the
feature was a silent no-op. Also fails instead of pruning when CONTEXT is
unreadable, rejects duplicate statement_ids, and skips multi-action
statements that add-permission cannot recreate.
An empty structure may read as "no change", which would drop the role
grant while leaving DeadLetterConfig in place.
merge_iam_policies names its policies per deployment, so a static list
left them behind and DeleteRole still failed. Only this scope's policies
are removed from a shared role.
LOG_LEVEL=DEBUG fell through to the default threshold.
@andres-nullplatform

Copy link
Copy Markdown
Contributor Author

Follow-up found while fixing the config path: several pre-existing readers use a lambda. (or networking.) level that does not exist in the scope-configuration schema, so they silently fall back to their defaults today.

  • scope/tofu/iam/setuplambda.execution_role_prefix, lambda.permissions_boundary
  • scope/scripts/create_iam_role, update_iam_rolelambda.use_dedicated_role, role_name_template, role_policies, dedicated_role_arn
  • deployment/scripts/merge_iam_policieslambda.role_policies
  • scope/build_contextnetworking.alb_listener_rule_capacity / _alert_threshold

providers.<category> is the provider's attributes object, so the correct paths have no such level (fetch_scope_configuration reads .state.tofu_state_bucket, .deployment.placeholder_image_uri, .agent.null_agent_layer_arn).

Not fixed here on purpose: correcting them would make configuration that is currently ignored take effect — lambda.role_policies would start attaching IAM policies, and the ALB capacity guard rail would start firing. That deserves its own change and its own testing. cleanup_role_inline_policies deliberately keeps reading the same path as iam/setup so both resolve the same role name.

The provider resolves against the scope NRN, so a value set higher up was
inherited by every lambda scope below it — entries can now name the scope
they belong to. Validation also moved ahead of the prune loop: rejecting
an entry after removing the statement it replaced left a live trigger
with no grant. A bare account id principal no longer churns every run.
Per-deployment policy names carry no scope marker, so they cannot be
attributed there. The header claimed otherwise.
Lambda also needs kms:GenerateDataKey on the key, which the scope cannot
derive from the target ARN.
Also documents the scope filter, which was in the schema but not here.
Without it a slug-scoped entry was filtered out and then pruned,
revoking a live trigger. The script now refuses to run without it.
A failed destroy leaves the function running; stripping them cost it
logs, ECR pulls and Secrets Manager access with nothing to restore them.
The grant is written first by design, so a failed config update left the
role with send access to an arbitrary ARN and no DLQ enabled.
The agent spawns the released worker image per action instead of
git-cloning this repo. Set worker_orchestrator=false to keep the old
channel. Also pins both modules to v7.0.0 — 7.0.0 is not a tag, and the
agent association was still on v6.19.1, which has no worker support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant