Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🟧 AWS Managed Service for Prometheus (AMP) Terraform Module

Provisions a secure-by-default Amazon Managed Service for Prometheus workspace together with its optional Alert Manager definition, rule group namespaces, fully-managed scrapers, cross-account resource policy, query logging, and retention/ingestion configuration — one auditable observability workspace from a single module call. Built for the AWS provider v6.x.

Terraform aws module type resources


🧩 Overview

  • 📦 One workspace, fully wired. Creates aws_prometheus_workspace plus everything that lives on it: Alert Manager definition, rule group namespaces, fully-managed scrapers, a resource-based (cross-account) policy, query logging configuration, and workspace (retention/ingestion) configuration.
  • 🔐 Encrypted at rest, always. Workspace metrics are encrypted with the AWS-owned AMP key by default; supply kms_key_arn to upgrade to a customer-managed CMK for full, CloudTrail-auditable key control.
  • 📜 Vended + query logging, opt-in but recommended. logging_configuration and query_logging_configuration both default to null only because they require a caller-supplied CloudWatch log group ARN — wire one from terraform-aws-cloudwatch-log-group for a regulated FI's audit trail.
  • 🤖 Fully-managed collectors. scrapers stands up aws_prometheus_scraper fully-managed collectors against an EKS cluster or a VPC source — no self-hosted Prometheus server required.
  • 🚨 Alerting and rules as code. alert_manager_definition and rule_group_namespaces accept raw YAML — prefer file/templatefile over inline heredocs so the YAML can be linted independently of Terraform.
  • 🔗 Cross-account sharing. resource_policy grants other accounts/services (e.g. Amazon Managed Grafana) least-privilege aps:QueryMetrics/aps:RemoteWrite access to this workspace.
  • 🏷️ Tags where they're allowed. var.tags flows to the workspace, each rule group namespace, and each scraper; merges with provider default_tags; the merged set is surfaced as tags_all.
  • 🌐 Region-inherited. No region variable — the caller's provider sets it. AMP is not a us-east-1 global service.

💡 Why it matters: Operational metrics are how a regulated FI notices trouble before it becomes an incident. A single secure-by-default AMP module keeps every workspace encrypted, its collectors correctly scoped to exactly one source, and its query/access trail auditable — so the observability layer itself never becomes the weak link.


❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!


🗺️ Where this fits in the family

terraform-aws-managed-prometheus is a containerized-workload observability module — it consumes networking, identity, encryption, and logging foundations, and its prometheus_endpoint/arn are consumed by workloads and downstream alarms.

flowchart LR
 kms["terraform-aws-kms<br/>CMK"]
 lg["terraform-aws-cloudwatch-log-group<br/>vended + query logs"]
 eks["terraform-aws-eks<br/>cluster"]
 vpc["terraform-aws-vpc<br/>subnets"]
 sg["terraform-aws-security-group"]
 role["terraform-aws-iam-role<br/>cross-account role_configuration"]
 pol["terraform-aws-iam-policy<br/>resource_policy document"]
 amp["terraform-aws-managed-prometheus"]
 grafana["Amazon Managed Grafana<br/>(query via resource_policy)"]
 ecs["terraform-aws-ecs-service /<br/>EKS workloads<br/>(remote_write)"]
 cw["CloudWatch Alarms<br/>(on AMP-scraped metrics)"]

 kms -->|"kms_key_arn"| amp
 lg -->|"log_group_arn (logging / query logging)"| amp
 eks -->|"scraper source.eks.cluster_arn"| amp
 vpc -->|"scraper subnet_ids"| amp
 sg -->|"scraper security_group_ids"| amp
 role -->|"scraper role_configuration ARNs"| amp
 pol -->|"resource_policy document"| amp
 amp -->|"prometheus_endpoint"| ecs
 amp -->|"resource_policy grants"| grafana
 amp -->|"scraped metrics"| cw

 style amp fill:#FF9900,color:#fff,stroke:#cc7a00,stroke-width:2px
Loading

🧬 What this module builds

flowchart TD
 subgraph mod["terraform-aws-managed-prometheus"]
 ws["aws_prometheus_workspace.this<br/>(keystone)<br/>alias + optional CMK + optional vended logging"]
 amd["aws_prometheus_alert_manager_definition.this<br/>optional singleton"]
 rgn["aws_prometheus_rule_group_namespace.this<br/>for_each rule_group_namespaces"]
 scr["aws_prometheus_scraper.this<br/>for_each scrapers"]
 rp["aws_prometheus_resource_policy.this<br/>optional singleton"]
 qlc["aws_prometheus_query_logging_configuration.this<br/>optional singleton"]
 wc["aws_prometheus_workspace_configuration.this<br/>optional singleton"]
 end

 ws --> amd
 ws --> rgn
 ws --> scr
 ws -. account-scoped sharing.-> rp
 ws --> qlc
 ws --> wc

 style ws fill:#FF9900,color:#fff,stroke:#cc7a00,stroke-width:2px
 style amd stroke-dasharray: 5 5
 style rp stroke-dasharray: 5 5
 style qlc stroke-dasharray: 5 5
 style wc stroke-dasharray: 5 5
Loading
Resource Count Created when
aws_prometheus_workspace.this 1 always (keystone)
aws_prometheus_alert_manager_definition.this 0 or 1 alert_manager_definition != null
aws_prometheus_rule_group_namespace.this 0..N one per rule_group_namespaces entry
aws_prometheus_scraper.this 0..N one per scrapers entry
aws_prometheus_resource_policy.this 0 or 1 resource_policy != null
aws_prometheus_query_logging_configuration.this 0 or 1 query_logging_configuration != null
aws_prometheus_workspace_configuration.this 0 or 1 workspace_configuration != null

✅ Provider / Versions

Requirement Version
Terraform >= 1.12.0
hashicorp/aws >= 6.0, < 7.0

The module declares only a required_providers block (providers.tf) and inherits the configured provider. There is no provider {} block and no credential variable — credentials resolve through the standard AWS chain at the root/pipeline level (env vars → SSO/shared credentials → assume_role → instance profile / IRSA → OIDC web identity).


🔑 Required IAM Permissions

Least-privilege actions the Terraform execution identity needs to manage this module.

Action Required for Notes
aps:CreateWorkspace, aps:DescribeWorkspace, aps:UpdateWorkspaceAlias, aps:DeleteWorkspace Workspace lifecycle Core CRUD on the keystone
aps:TagResource, aps:UntagResource, aps:ListTagsForResource Tagging Workspace, rule group namespaces, scrapers
aps:PutAlertManagerDefinition, aps:DescribeAlertManagerDefinition, aps:DeleteAlertManagerDefinition Alert Manager definition Only with alert_manager_definition
aps:CreateRuleGroupsNamespace, aps:DescribeRuleGroupsNamespace, aps:PutRuleGroupsNamespace, aps:DeleteRuleGroupsNamespace Rule group namespaces Only with rule_group_namespaces
aps:CreateScraper, aps:DescribeScraper, aps:DeleteScraper Scrapers Only with scrapers
iam:CreateServiceLinkedRole (for aps.amazonaws.com) Scraper's auto-created discovery/collection role First scraper in the account only
iam:PassRole (on role_configuration ARNs) Cross-account scraper write Only with scrapers[*].role_configuration
aps:PutResourcePolicy, aps:DescribeResourcePolicy, aps:DeleteResourcePolicy Resource-based (cross-account) policy Only with resource_policy
aps:PutQueryLoggingConfiguration (a.k.a. aps:CreateLoggingConfiguration), aps:DescribeQueryLoggingConfiguration, aps:DeleteQueryLoggingConfiguration Query logging configuration Only with query_logging_configuration
aps:PutWorkspaceConfiguration (a.k.a. aps:CreateWorkspaceConfiguration), aps:DescribeWorkspaceConfiguration Retention / label-set limits Only with workspace_configuration
logs:CreateLogDelivery, logs:PutResourcePolicy (on the target log group) Vended workspace logging and query logging delivery Scoped to the supplied log group ARN
kms:DescribeKey, kms:CreateGrant Validate/grant a caller-supplied CMK Only when kms_key_arn is set
eks:DescribeCluster Read back the source EKS cluster referenced by a scraper Only with scrapers[*].source.eks

🔒 Scope aps:* actions to the workspace ARN pattern (arn:aws:aps:<region>:<account>:workspace/*) once the workspace exists; the initial aps:CreateWorkspace call cannot itself be resource-scoped.

ℹ️ Service-linked role. AMP scrapers rely on a service-linked (or AMP-managed) role to discover and collect metrics — the executing identity needs iam:CreateServiceLinkedRole the first time a scraper is created in the account/region.


📋 AWS Prerequisites

  • Service-linked role for scrapers. A managed scraper's discovery/collection role is created automatically on first use; grant iam:CreateServiceLinkedRole up front so the first scrapers entry doesn't fail.
  • EKS networking reachability. A scraper's collector must have network line-of-sight (VPC routing/peering/Transit Gateway) to the source EKS cluster's API server and the pods/nodes it scrapes, and the cluster's aws-auth ConfigMap (or EKS access entries) must grant the scraper's role read access — see the AMP user guide's "Configure aws-auth" step.
  • KMS key policy (when kms_key_arn is set). The CMK's policy must allow the AMP service principal (aps.amazonaws.com) to use the key on the workspace's behalf; if resource_policy grants cross-account access, those principals also need a KMS grant.
  • CloudWatch log group prerequisite. logging_configuration.log_group_arn and query_logging_configuration.log_group_arn must reference a log group that already exists, with an ARN carrying the :* (all-streams) suffix — the API rejects the bare-group ARN form.
  • Region. Provider-inherited; there is no region variable. AMP is not a us-east-1 global service.
  • Cost / quotas. retention_period_in_days directly drives storage cost — size it against a records-retention requirement, not convenience. Default AMP quotas cover workspaces per account/region, active time series per workspace, and scrapers per workspace — check the AMP service quotas page before provisioning at scale.

📁 Module Structure

terraform-aws-managed-prometheus/
├── providers.tf # required_providers (aws >= 6.0, < 7.0); no provider block
├── variables.tf # alias → kms/logging → alert manager → child collections → resource policy → query logging → workspace config → tags
├── main.tf # aws_prometheus_workspace.this + alert manager, rule groups, scrapers, resource policy, query logging, workspace config
├── outputs.tf # id + arn + prometheus_endpoint + child-collection maps + tags_all
├── README.md # this file
└── SCOPE.md # in/out-of-scope, IAM permissions, prerequisites, gotchas

⚙️ Quick Start

Smallest working call — an aliased workspace, encrypted by the AWS-owned key:

module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  tags = {
    Environment = "prod"
    CostCenter  = "1234"
  }
}

🔌 Cross-Module Contract

Consumes

Input Type Source module
kms_key_arn string (KMS key ARN) terraform-aws-kms
logging_configuration.log_group_arn string (CloudWatch log group ARN, :* suffix) terraform-aws-cloudwatch-log-group
query_logging_configuration.log_group_arn string (CloudWatch log group ARN, :* suffix) terraform-aws-cloudwatch-log-group
scrapers[*].source.eks.cluster_arn string (EKS cluster ARN) terraform-aws-eks
scrapers[*].source.{eks,vpc}.subnet_ids list(string) terraform-aws-vpc
scrapers[*].source.{eks,vpc}.security_group_ids list(string) terraform-aws-security-group
scrapers[*].role_configuration.*_role_arn string (IAM role ARN) terraform-aws-iam-role
resource_policy string (JSON policy document) terraform-aws-iam-policy document, or inline aws_iam_policy_document

Emits

Output Description Consumed by
id Workspace ID (ws-<uuid>) AWS CLI / console cross-referencing
arn Workspace ARN — cross-resource reference type terraform-aws-iam-policy (aps:* scoping), scrapers, resource policy, query logging config
alias Workspace alias, or null Dashboards / inspection
prometheus_endpoint Prometheus-compatible remote_write/query endpoint URL EKS/ECS workload remote_write, Amazon Managed Grafana data source
kms_key_arn CMK ARN, or null for the AWS-owned key Governance/audit
rule_group_namespaces Map of namespace name → { id, arn } Inspection
scrapers Map of scraper key → { id, arn, role_arn } role_arn feeds EKS aws-auth/access-entry wiring
alert_manager_definition_workspace_id Workspace ID, or null when not supplied Inspection
resource_policy_revision_id Revision ID, or null when not supplied Drift detection
query_logging_configuration_workspace_id Workspace ID, or null when not supplied Inspection
workspace_configuration_retention_period_in_days Effective retention, or null Governance/audit
tags_all All tags incl. provider default_tags Governance/audit

📚 Example Library

1 · Minimal — secure defaults (AWS-owned key, unaliased)
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"
  # workspace is encrypted at rest by the AWS-owned AMP key by default
}
2 · Aliased workspace
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"
  alias  = "casey-prod-metrics"
}
3 · Customer-managed KMS CMK wired from terraform-aws-kms
module "amp_kms" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-kms?ref=v1.0.0"
  alias  = "casey/amp-metrics"
  # key policy must allow aps.amazonaws.com to use the key on the workspace's behalf
}

module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias       = "casey-prod-metrics"
  kms_key_arn = module.amp_kms.arn # upgrade from AWS-owned key to a CMK; FORCE-NEW if changed later
}
4 · Tags (merge with provider default_tags)
# Caller's provider block owns default_tags; the module never sets it.
provider "aws" {
  default_tags { tags = { Owner = "platform", ManagedBy = "terraform" } }
}

module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  tags = {
    Environment = "prod" # resource tag — wins over default_tags on key conflict
    DataClass   = "operational-metrics"
  }
}

# module.amp.tags_all == { Owner, ManagedBy, Environment, DataClass }
5 · Vended workspace logging + query logging (secure-baseline opt-in), wired from terraform-aws-cloudwatch-log-group
module "amp_logs" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-cloudwatch-log-group?ref=v1.0.0"
  name   = "/casey/amp/workspace"
}

module "amp_query_logs" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-cloudwatch-log-group?ref=v1.0.0"
  name   = "/casey/amp/query-logs"
}

module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  logging_configuration = {
    log_group_arn = module.amp_logs.arn_with_suffix # MUST carry the ":*" suffix
  }

  query_logging_configuration = {
    log_group_arn = module.amp_query_logs.arn_with_suffix
    filters       = [{ qsp_threshold = 1000 }]
  }
}
6 · Secure-by-default opt-out — no logging configured
module "amp_dev" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-dev-metrics"
  # logging_configuration and query_logging_configuration left null (default) —
  # acceptable for a non-production sandbox; document the exception for prod.
}
7 · Alert Manager definition from a linted YAML file
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  # Prefer file/templatefile over an inline heredoc so the YAML can be
  # linted (yamllint) independently of Terraform — the AMP API only validates
  # this at apply time, not at terraform validate/plan.
  alert_manager_definition = file("${path.root}/alertmanager.yaml")
}
8 · Rule group namespaces (recording + alerting rules)
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  rule_group_namespaces = {
    recording-rules = {
      data = file("${path.root}/rules/recording-rules.yaml")
    }
    alerting-rules = {
      data = file("${path.root}/rules/alerting-rules.yaml")
      tags = { RuleClass = "alerting" }
    }
  }
}
9 · EKS-sourced fully-managed scraper, wired from terraform-aws-eks / terraform-aws-vpc
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  scrapers = {
    eks-prod = {
      scrape_configuration = file("${path.root}/scrape-config.yaml")
      source = {
        eks = {
          cluster_arn = module.eks.arn
          subnet_ids  = module.vpc.private_subnet_ids
        }
      }
    }
  }

  # A managed scraper adds an "AMPAgentlessScraper" tag to this workspace on
  # create — include the same key here to avoid a perpetual tag-drift diff.
  tags = { AMPAgentlessScraper = "" }
}
10 · VPC-sourced scraper (non-EKS Prometheus-compatible metrics)
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  scrapers = {
    ecs-service-discovery = {
      scrape_configuration = file("${path.root}/scrape-config-vpc.yaml")
      source = {
        vpc = {
          security_group_ids = [module.scraper_sg.id]
          subnet_ids         = module.vpc.private_subnet_ids
        }
      }
    }
  }
}
11 · Cross-account scraper write (source account → target workspace)
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-central-metrics"

  scrapers = {
    remote-eks = {
      scrape_configuration = file("${path.root}/scrape-config.yaml")
      source = {
        eks = {
          cluster_arn = "arn:aws:eks:us-east-2:111111111111:cluster/source-cluster"
          subnet_ids  = ["subnet-aaaa", "subnet-bbbb"]
        }
      }
      destination = {
        amp = { workspace_arn = "arn:aws:aps:us-east-2:222222222222:workspace/ws-target" }
      }
      role_configuration = {
        source_role_arn = module.scraper_source_role.arn # in the source (scraper) account
        target_role_arn = "arn:aws:iam::222222222222:role/amp-cross-account-write"
      }
    }
  }
}
# NOTE: the target role and target workspace must be in the same account;
# the source scraper and target workspace must be in the same Region.
12 · Resource-based policy — cross-account query access (e.g. Amazon Managed Grafana)
data "aws_iam_policy_document" "amp_grafana_access" {
  statement {
    effect = "Allow"
    principals {
      type        = "Service"
      identifiers = ["grafana.amazonaws.com"]
    }
    actions = [
      "aps:QueryMetrics",
      "aps:GetSeries",
      "aps:GetLabels",
      "aps:GetMetricMetadata",
    ]
    resources = [module.amp.arn] # must match the workspace's own ARN
  }
}

module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias           = "casey-prod-metrics"
  resource_policy = data.aws_iam_policy_document.amp_grafana_access.json
}
13 · Retention + per-label-set ingestion limits (workspace_configuration)
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-prod-metrics"

  workspace_configuration = {
    retention_period_in_days = 90 # size against a records-retention requirement, not convenience
    limits_per_label_set = [
      { label_set = {}, max_series = 50000 }, # default bucket
      { label_set = { env = "prod" }, max_series = 400000 },
      { label_set = { env = "dev" }, max_series = 100000 },
    ]
  }
}
14 · for_each pattern — one workspace per environment
locals {
  environments = toset(["dev", "staging", "prod"])
}

module "amp" {
  for_each = local.environments
  source   = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias = "casey-${each.key}-metrics"
  tags  = { Environment = each.key }
}
15 · Import an existing workspace
import {
  to = module.amp.aws_prometheus_workspace.this
  id = "ws-C6DCB907-F2D7-4D96-957B-66691F865D8B"
}
16 · End-to-end composition — CMK + logging + rules + EKS scraper + Grafana sharing
# 1) Customer-managed CMK for workspace encryption
module "amp_kms" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-kms?ref=v1.0.0"
  alias  = "casey/amp-metrics"
}

# 2) Log groups for vended + query logging
module "amp_logs" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-cloudwatch-log-group?ref=v1.0.0"
  name   = "/casey/amp/workspace"
}

module "amp_query_logs" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-cloudwatch-log-group?ref=v1.0.0"
  name   = "/casey/amp/query-logs"
}

# 3) Cross-account/service sharing document
data "aws_iam_policy_document" "amp_grafana_access" {
  statement {
    effect = "Allow"
    principals {
      type        = "Service"
      identifiers = ["grafana.amazonaws.com"]
    }
    actions   = ["aps:QueryMetrics", "aps:GetSeries", "aps:GetLabels", "aps:GetMetricMetadata"]
    resources = [module.amp.arn]
  }
}

# 4) This module — encrypted workspace with logging, rules, an EKS scraper, and sharing
module "amp" {
  source = "git::https://github.com/microsoftexpert/terraform-aws-managed-prometheus?ref=v1.0.0"

  alias       = "casey-prod-metrics"
  kms_key_arn = module.amp_kms.arn

  logging_configuration = {
    log_group_arn = module.amp_logs.arn_with_suffix
  }

  query_logging_configuration = {
    log_group_arn = module.amp_query_logs.arn_with_suffix
  }

  rule_group_namespaces = {
    recording-rules = { data = file("${path.root}/rules/recording-rules.yaml") }
  }

  scrapers = {
    eks-prod = {
      scrape_configuration = file("${path.root}/scrape-config.yaml")
      source = {
        eks = {
          cluster_arn = module.eks.arn
          subnet_ids  = module.vpc.private_subnet_ids
        }
      }
    }
  }

  workspace_configuration = {
    retention_period_in_days = 90
  }

  resource_policy = data.aws_iam_policy_document.amp_grafana_access.json

  tags = {
    Environment         = "prod"
    AMPAgentlessScraper = "" # matches the scraper's auto-applied workspace tag
  }
}

# 5) Consume the endpoint downstream (e.g. EKS ADOT collector remote_write config)
output "amp_remote_write_endpoint" {
  value = "${module.amp.prometheus_endpoint}api/v1/remote_write"
}

📥 Inputs

Name Type Default Description
alias string null Workspace alias (human-readable name).
kms_key_arn string (ARN) null Customer-managed CMK for at-rest encryption; null uses the AWS-owned key. FORCE-NEW.
logging_configuration object({ log_group_arn }) null Vended workspace logging; log group ARN must carry the :* suffix.
alert_manager_definition string (YAML) null Alert Manager configuration.
rule_group_namespaces map(object({...})) {} Rule group namespaces keyed by name; each taggable.
scrapers map(object({...})) {} Fully-managed collectors keyed by caller name; exactly one of source.eks/source.vpc.
resource_policy string (JSON) null Cross-account/service resource-based policy document.
resource_policy_timeouts object({ create, update, delete }) {} Operation timeouts for the resource policy.
query_logging_configuration object({ log_group_arn, filters }) null Query logging; log group ARN must carry the :* suffix.
query_logging_configuration_timeouts object({ create, update, delete }) {} Operation timeouts for query logging configuration.
workspace_configuration object({...}) null Retention and per-label-set ingestion limits.
workspace_configuration_timeouts object({ create, update }) {} Operation timeouts for the workspace configuration (no delete — no destroy-side API call).
tags map(string) {} Tags for the workspace, rule group namespaces, and scrapers (merge with default_tags).

See variables.tf for full heredoc schemas and validation rules.


🧾 Outputs

Name Description
id Workspace ID (ws-<uuid>).
arn Workspace ARN — cross-resource reference type.
alias Workspace alias, or null.
prometheus_endpoint Prometheus-compatible remote_write/query endpoint URL.
kms_key_arn CMK ARN, or null for the AWS-owned key.
rule_group_namespaces Map of namespace name → { id, arn }.
scrapers Map of scraper key → { id, arn, role_arn }.
alert_manager_definition_workspace_id Workspace ID, or null when not supplied.
resource_policy_revision_id Revision ID, or null when not supplied.
query_logging_configuration_workspace_id Workspace ID, or null when not supplied.
workspace_configuration_retention_period_in_days Effective retention in days, or null.
tags_all All tags incl. provider default_tags.

🧠 Architecture Notes

  • ARN format: arn:aws:aps:<region>:<account-id>:workspace/<workspace_id>. id is the bare workspace identifier (ws-<uuid>); arn is the cross-resource reference type for IAM policy Resource elements, scraper destination.amp.workspace_arn, and the resource_policy document's Resource element (must match this workspace's ARN exactly).
  • Force-new fields: kms_key_arn is FORCE-NEW — changing the CMK replaces the workspace (and its metrics history). A scraper's source (EKS or VPC configuration) is also effectively FORCE-NEW: per the provider, changing it deletes the current scraper and creates a new one, with no in-place update path.
  • tagstags_alldefault_tags: var.tags flows to aws_prometheus_workspace.this.tags, aws_prometheus_rule_group_namespace.this.tags, and aws_prometheus_scraper.this.tags. aws_prometheus_alert_manager_definition, aws_prometheus_resource_policy, aws_prometheus_query_logging_configuration, and aws_prometheus_workspace_configuration are not taggable in this provider version. tags_all is the computed merge of resource tags over provider default_tags, with resource tags winning on key conflict; default_tags is the caller's provider-block concern.
  • Raw-YAML fields validate only at apply time. alert_manager_definition and rule_group_namespaces[*].data are opaque strings to Terraform — a malformed YAML body or indentation typo passes validate/plan cleanly and only fails against the live AMP API at apply. Prefer file/templatefile over an inline heredoc so the source can be linted (yamllint) independently, and catch errors in CI before apply.
  • Live-schema vs. published-docs discrepancies (provider v6.54.0) — verified directly against terraform providers schema -json, not just the rendered docs page:
  • aws_prometheus_scraper does expose tags/tags_all, even though the Registry doc's prose omits them from the Argument/Attribute Reference — this module tags scrapers with var.tags.
  • aws_prometheus_resource_policy's published policy_status attribute does not exist in the resolved schema (only policy_document, region, revision_id, workspace_id); this module does not output it.
  • aws_prometheus_alert_manager_definition, aws_prometheus_resource_policy, aws_prometheus_query_logging_configuration, and aws_prometheus_workspace_configuration have no top-level id distinct from workspace_id — they are framework-based resources keyed by workspace_id. Outputs reference workspace_id/revision_id/retention_period_in_days rather than a nonexistent .id.
  • Re-verify against a fresh schema dump on every provider bump rather than trusting the docs page for this resource family.
  • Eventual consistency: a scraper's discovery/collection service-linked role can take a few seconds to propagate on first creation in an account; a transient AccessDenied on scraper creation often resolves on retry.
  • Destroy ordering: the workspace's sub-resources (Alert Manager definition, rule group namespaces, scrapers, resource policy, query logging, workspace configuration) are all torn down with/before the workspace automatically — there is no ENI/NAT-style destroy-ordering hazard here.
  • us-east-1 globals: N/A. AMP is a regional service.

🧱 Design Principles

Secure-by-default posture and every opt-out, explicitly:

Posture Default Opt-out
Encryption at rest AWS-owned AMP key (always on); CMK when kms_key_arn set supply / omit kms_key_arn
Vended workspace logging opt-in (logging_configuration = null) — cannot be forced on without a caller-supplied log group ARN supply logging_configuration.log_group_arn (recommended for prod)
Query logging (auditability) opt-in (query_logging_configuration = null) for the same reason supply query_logging_configuration.log_group_arn (recommended for prod)
Retention platform default (subject to account quotas) unless configured set workspace_configuration.retention_period_in_days
Cross-account/service sharing none (resource_policy = null) supply a least-privilege resource_policy scoped to this workspace's ARN
Region provider-inherited use a provider alias in providers = {} if needed

AMP workspace metrics are always encrypted at rest, even with kms_key_arn = null. Supplying a CMK upgrades to full, independently revocable key control — recommended for a regulated FI's operational metrics. Document each opt-out in your root module so reviewers can see what was loosened.

Other principles:

  • One composite, one keystone. The workspace owns only the resources meaningless without it. The EKS cluster, subnets, security groups, IAM roles, KMS key, and CloudWatch log groups it wires to are all referenced by ARN — keeping blast radius to the workspace and its direct sub-resources.
  • Conditional singletons via guarded for_each, never count — Alert Manager definition, resource policy, query logging configuration, and workspace configuration each materialize only when the caller supplies the corresponding object.
  • for_each, never count, for rule group namespaces and scrapers, keyed by stable caller strings so reorders don't churn the plan.
  • Exactly one scraper source enforced at plan time. A variables.tf validation block rejects a scrapers entry that sets both source.eks and source.vpc, or neither, rather than deferring to the API's own rejection at apply.
  • Primary outputs id + arn, plus prometheus_endpoint, child-collection maps, and tags_all.

🚀 Runbook

# Validate without backend or credentials
terraform init -backend=false
terraform validate
terraform fmt -check

plan / apply require valid AWS credentials (profile / SSO / OIDC) resolved through the standard provider chain, a configured Region, and the aps:* actions listed above. When kms_key_arn is set, the CMK key policy must already allow aps.amazonaws.com.

⚠️ Always pin the module source with ?ref=v1.0.0 (an immutable tag), never a branch.


🧪 Testing

  • terraform init -backend=false && terraform validate — schema + reference integrity.
  • terraform fmt -check — canonical formatting.
  • terraform plan against a sandbox account to confirm the workspace, and any configured Alert Manager definition, rule group namespaces, scrapers, resource policy, query logging, and workspace configuration materialize as expected.
  • Assert module.<name>.arn, prometheus_endpoint, scrapers, and tags_all in your root-module test harness.
  • Negative test: a scrapers entry with both source.eks and source.vpc (or neither) should fail validation at plan time.
  • Lint alert_manager_definition / rule_group_namespaces[*].data YAML files with yamllint in CI before apply — Terraform cannot catch a YAML error here.

💬 Example Output

module.amp.aws_prometheus_workspace.this: Creation complete after 45s [id=ws-C6DCB907-F2D7-4D96-957B-66691F865D8B]
module.amp.aws_prometheus_rule_group_namespace.this["recording-rules"]: Creation complete
module.amp.aws_prometheus_scraper.this["eks-prod"]: Still creating... [10s elapsed]
module.amp.aws_prometheus_scraper.this["eks-prod"]: Creation complete after 3m12s

Outputs:
alias = "casey-prod-metrics"
arn = "arn:aws:aps:us-east-2:123456789012:workspace/ws-C6DCB907-F2D7-4D96-957B-66691F865D8B"
id = "ws-C6DCB907-F2D7-4D96-957B-66691F865D8B"
prometheus_endpoint = "https://aps-workspaces.us-east-2.amazonaws.com/workspaces/ws-C6DCB907-F2D7-4D96-957B-66691F865D8B/"
scrapers = { "eks-prod" = { "arn" = "arn:aws:aps:us-east-2:123456789012:scraper/s-0123abc-...", "id" = "s-0123abc-0000-0123-a000-000000000000", "role_arn" = "arn:aws:iam::123456789012:role/aws-service-role/..." } }
tags_all = { "Environment" = "prod" }

🔍 Troubleshooting

Symptom Likely cause Fix
Scraper creation fails with AccessDenied on first use in the account iam:CreateServiceLinkedRole missing, or the service-linked role hasn't propagated yet Grant the permission; retry after a short wait
Scraper stuck in CREATING / fails to collect metrics EKS cluster not reachable from the supplied subnets, or aws-auth/access entries don't grant the scraper's role read access Confirm VPC routing/peering and follow the AMP "Configure aws-auth" step
ValidationException: exactly one of eks or vpc at apply Both source.eks and source.vpc set, or neither Set exactly one — this module's own variables.tf validation should already catch this at plan time
Changing a scraper's EKS cluster or subnets replaces it unexpectedly source is effectively FORCE-NEW — no in-place update path Expected provider behavior; plan for a brief collection gap during replacement
Perpetual tag diff on the workspace A managed scraper auto-applies an AMPAgentlessScraper tag that isn't in var.tags Add AMPAgentlessScraper = "" to tags (preferred), or lifecycle { ignore_changes = [tags] } on the workspace
logging_configuration / query_logging_configuration rejected at apply Log group ARN missing the :* suffix, or the log group doesn't exist yet Use the log group's arn_with_suffix output (from terraform-aws-cloudwatch-log-group); ensure it's created first
Alert Manager definition or rule group rejected at apply, terraform validate was clean Raw YAML string had a whitespace/indentation error — only the AMP API validates it Lint the YAML source with yamllint before apply; use file/templatefile instead of an inline heredoc
Cross-account query/write access denied despite a resource_policy Policy Resource doesn't match the workspace's own ARN exactly, or (with a CMK) the principal lacks a KMS grant Match Resource to module.amp.arn; add a KMS grant when kms_key_arn is set
terraform validate passes but apply fails with AccessDenied Executing IAM role missing an aps:*/iam:PassRole/kms:* action from the Required IAM Permissions table Add the specific action least-privilege; do not broaden to aps:* blanket
Tag drift from a rule group namespace or scraper tag on every plan A tag key also set (with a different value) in provider default_tags Let resource tags win, or remove the overlapping key from default_tags

🔗 Related Docs


🧡 "Infrastructure as Code should be standardized, consistent, and secure."