diff --git a/skills/investigation-cost-guardrail/CHANGELOG.md b/skills/investigation-cost-guardrail/CHANGELOG.md index 436277f..8e1c45b 100644 --- a/skills/investigation-cost-guardrail/CHANGELOG.md +++ b/skills/investigation-cost-guardrail/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +## 2.1.0 + + - Replace every hardcoded rate with a live AWS Price List Query API lookup, resolved for the workload region at estimation time. The skill no longer carries baseline rates of any kind. + - Add `references/pricing-reference.md` holding the exact call patterns: which filter field each operation uses (`operation` vs `usagetype`), the exact filter value, region scoping, the `usagetype` region-prefix map and its us-east-1 exceptions, S3 request tiers and the three meters S3 Select bills on, and the filtered cross-region data transfer lookup. + - `SKILL.md` loads the reference on the first operation Layer 2 classifies as PAID and reuses it for the rest of the investigation. + - Layer 2 now records filter fields and formulas instead of rates, covering CloudWatch Logs Insights, `GetMetricData`, `GetInsightRuleReport`, Live Tail, PromQL samples scanned, X-Ray, Athena, DynamoDB, and S3 requests and Select. + + ## 2.0.0 - Expand the scope from a fixed CloudWatch/X-Ray/CloudTrail set to all AWS service with the agent's own tools; `aws-services` metadata changes to `All`. diff --git a/skills/investigation-cost-guardrail/README.md b/skills/investigation-cost-guardrail/README.md index c34006f..49922e6 100644 --- a/skills/investigation-cost-guardrail/README.md +++ b/skills/investigation-cost-guardrail/README.md @@ -10,7 +10,7 @@ The agent decides whether to load a skill based on description matching. To guar 1. **Classify:** Determine if the operation is free or paid -2. **Estimate:** Calculate expected cost using known formulas or heuristics +2. **Estimate:** Resolve the live per-Region rate, then apply the operation's formula or heuristic 3. **Check budget:** Compare estimated cost against per-investigation budget @@ -47,7 +47,7 @@ Rather than hardcoding every free/paid operation across 200+ AWS services, this │ • Covers ANY current or future AWS service │ ├─────────────────────────────────────────────────┤ │ Layer 2: Known-Paid Registry │ -│ • Exact pricing formulas for high-cost ops │ +│ • Live per-Region rate lookup + formulas │ │ • Extensible by operator configuration │ │ • Athena, DynamoDB, S3, X-Ray, SageMaker... │ ├─────────────────────────────────────────────────┤ @@ -65,7 +65,7 @@ Even if AWS launches a new service tomorrow, the heuristic rules will correctly | Tool | Classification | Cost Model | Guardrail | | --- | --- | --- | --- | -| get_prometheus_metrics | **PAID** | $0.01 / 1,000 metrics×periods (same billing meter as CloudWatch GetMetricData) | Track series × datapoints per call | +| get_prometheus_metrics | **PAID** | Billed per sample scanned — rate resolved from a live `CW:PromQL:SamplesScanned` usagetype lookup | Track samples scanned per call | | use_aws | **VARIABLE** | Depends on operation — apply Layers 1–3 | Full heuristic pipeline | | use_azure | **FREE** | Azure Reader role, no per-call billing | Track count only | | grafana_query_prometheus | **CAUTION** | Depends on Grafana data source billing model | Track count, warn at 50+ | @@ -83,25 +83,30 @@ Even if AWS launches a new service tomorrow, the heuristic rules will correctly | Classification | Rule | | --- | --- | -| **FREE** | Verb is Describe, List, Get, Lookup, Check, Validate, Tag: returns metadata only | +| **FREE** | Verb is Describe, List, Get, Lookup, Check, Validate, Tag, Untag: returns metadata only | | **PAID** | Verb contains Query, Scan, Execute, Invoke, Insights: processes or scans data | | **CAUTION** | Paginated List/Describe with broad scope | ### Layer 2: Known-Paid Registry -| Service | Operation | Cost Formula | -| --- | --- | --- | -| CloudWatch Logs | StartQuery | $0.005/GB scanned | -| CloudWatch Logs | StartLiveTail | $0.01/minute | -| CloudWatch | GetMetricData / PromQL | $0.01/1,000 metrics×periods | -| X-Ray | GetTraceSummaries, BatchGetTraces | $0.50/1M traces | -| Athena | StartQueryExecution | $5.00/TB scanned | -| DynamoDB | Scan | ~$0.25/1M RCU. **BLOCKED unless approved** | -| S3 | GetObject | $0.0004/1K requests + $0.09/GB transfer | -| S3 | SelectObjectContent | $0.002/GB scanned + $0.0007/GB returned | -| SageMaker | InvokeEndpoint | **BLOCKED: requires explicit approval** | -| Lambda | Invoke | **BLOCKED unless user explicitly requests** | -| Kinesis | GetRecords | $0.015/1M records | +The registry holds no rates of its own. It records, per operation, which Pricing API filter field to query (`operation` or `usagetype`), the exact filter value, and the formula the resolved rate feeds into. `usagetype` and `operation` are different filter fields, and the value is never derived from the AWS API operation name — both are stated explicitly per operation. + +| Service | Operation | Rate resolved via | Cost Formula | +| --- | --- | --- | --- | +| CloudWatch Logs | StartQuery | `operation` | `scan_gb × rate` | +| CloudWatch Logs | StartLiveTail | `operation` | Duration-based | +| CloudWatch | GetMetricData | `operation` | `(metrics × periods) × rate` | +| CloudWatch | GetInsightRuleReport | `usagetype` | `metrics_requested × rate` | +| CloudWatch | PromQL (`get_prometheus_metrics`) | `usagetype` | `samples_scanned × rate` | +| X-Ray | GetTraceSummaries, BatchGetTraces | `operation` | `traces × rate` | +| Athena | StartQueryExecution | `usagetype` | `scan_tb × rate`, min 10MB | +| DynamoDB | Scan | `usagetype` | `RCU × rate`. **BLOCKED unless approved** | +| DynamoDB | Query | `usagetype` | `RCU × rate` | +| S3 | GetObject | `usagetype` (Tier2) | Per request | +| S3 | ListObjects, PutObject, CopyObject | `usagetype` (Tier1) | Per request | +| S3 | SelectObjectContent | `usagetype` (3 meters) | Bytes scanned + bytes returned + request | +| SageMaker | InvokeEndpoint | — | **BLOCKED: requires explicit approval** | +| Lambda | Invoke | — | **BLOCKED unless user explicitly requests** | **Operators can extend this registry:** @@ -110,6 +115,21 @@ Even if AWS launches a new service tomorrow, the heuristic rules will correctly ``` +### Regional Rate Resolution + +Rates vary by AWS Region, so every rate is resolved live for the workload's Region. `references/pricing-reference.md` holds the call patterns that do it. + +| Step | What happens | +| --- | --- | +| 1. Region | Derived from the resource ARN — never the agent's runtime Region, never a default | +| 2. Lookup | `pricing:GetProducts`. `operation` lookups pass the workload Region as a `regionCode` filter; `usagetype` lookups prepend the workload-Region prefix | +| 3. Cache | Keyed on `(service, operation, region)` — one lookup per service and Region per investigation | + + +There is no fallback rate. If the lookup cannot be resolved exactly, the skill halts rather than estimating: it will not improvise a rate from memory or training data. The operator is offered the choice to re-check the filter field, value, and Region prefix, use a free alternative, or have the lookup gap reported. + +It loads once, on the first operation classified as PAID, and is reused for the rest of the investigation. Investigations that touch only metadata or third-party tools do not load it. + ### Layer 3: Response Validation (Self-Learning) After execution, the skill checks response fields for metered indicators: @@ -123,11 +143,10 @@ After execution, the skill checks response fields for metered indicators: | ContentLength > 100MB | Large object transfer | | NextToken after 10+ pages | Pagination runaway | -If a previously-unclassified operation returns metered fields, it is reclassified as paid for the remainder of the investigation. ## Budget Enforcement -The skill maintains a **running cost accumulator** throughout each investigation using the scratchpad: +The skill maintains a running cost accumulator throughout each investigation using the scratchpad: ``` 📋 INVESTIGATION BUDGET STATUS @@ -173,11 +192,7 @@ When the next operation would exceed the budget: ## Cross-Region Detection -When the target region differs from the Agent Space region, the skill adds estimated data transfer cost ($0.02/GB) based on return size: - -- Aggregation queries: ~KB (negligible) -- Raw log/trace fetches: up to 100% of matched bytes -- Unknown: 15% of scan volume as upper bound (flagged ⚠️) +When the target region differs from the Agent Space region, the skill adds a data transfer cost. The rate is looked up live for that specific source → destination route and cached per route, because inter-Region rates vary widely by geography (roughly $0.01–$0.15/GB depending on source Region) — there is no flat rate to assume. ## Cost Reduction Suggestions @@ -191,10 +206,13 @@ When halting, the skill always suggests cost-efficient alternatives: | dynamodb:Scan | dynamodb:Query with key condition | ~100% | | athena:StartQueryExecution (full) | Add partition filter in WHERE | 90%+ | | xray:GetTraceSummaries (broad) | Narrow time + add filter expression | 90%+ | +| s3:GetObject (large) | s3:SelectObjectContent with SQL filter | Variable | | Broad time window | Narrow to ±30 min around incident | 90%+ | ## Scenarios +The dollar figures below are illustrative output. Every one of them is computed from a rate the skill resolved live for the workload's Region at estimation time; none is a rate published by this skill. + **Scenario A: Scoped investigation, within budget** ``` @@ -241,19 +259,24 @@ Agent attempts: athena:StartQueryExecution (full scan, no WHERE clause) ``` -**Scenario D: PromQL broad query warned** +**Scenario D: PromQL broad query flagged** ``` Agent attempts: get_prometheus_metrics (no label filter, 7d range, 60s step) - ⚠️ WARN: PromQL query would fetch ~5M metrics×periods - 💰 Estimated cost: $50.00 + ⚠️ FLAGGED: query hit the 500-series cap + ~5.04M samples scanned (500 series × 10,080 datapoints at 60s step) + 💰 ~$0.05 at the live CW:PromQL:SamplesScanned rate + + The series cap means this query cost the maximum it could for + this range and step, and the returned data is truncated — so the + result is both incomplete and needlessly broad. 💡 Suggestions: → Add label filters to reduce series count → Use topk(10, ...) to cap series - → Increase step to 300s - → Narrow time range to 1h + → Increase step to 300s (5× cheaper) + → Narrow time range to 1h (168× cheaper) ``` @@ -287,8 +310,32 @@ Add the skill to your Agent Space and adjust the threshold to match your organiz **Option B:** Download the `.zip` directly from the [repository](https://github.com/aws/tools-for-devops-agent/tree/main/skills/investigation-cost-guardrail) and upload it as a skill in your Agent Space. +### Required IAM Permissions + +The skill calls the AWS Price List Query API to resolve per-Region rates. Grant the role your Agent Space assumes: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": "pricing:GetProducts", + "Resource": "*" + } + ] +} +``` + +`Resource` is `*` because the Price List API returns public pricing data. The API is free and read-only. + +The call is always sent to `us-east-1`, so a Region-scoped tool policy needs to allow `us-east-1`. + +Your Agent Space tool policy must also permit the call. This permission is required, not optional: the skill carries no baseline rates to fall back on, so if the lookup is unavailable it halts before the paid operation instead of estimating. + + ## Known Limitations - **Budget is scoped to a single investigation:** each investigation starts with a fresh budget; cumulative tracking across multiple investigations at the agent space level is not currently supported. - **Skill-halted investigations show "Completed" status:** halt reason is only visible in the investigation output. - +- **A paid operation cannot be estimated without a successful rate lookup:** there is no fallback rate, so a missing `pricing:GetProducts` permission halts that operation. \ No newline at end of file diff --git a/skills/investigation-cost-guardrail/SKILL.md b/skills/investigation-cost-guardrail/SKILL.md index 92750d8..7d5b880 100644 --- a/skills/investigation-cost-guardrail/SKILL.md +++ b/skills/investigation-cost-guardrail/SKILL.md @@ -3,12 +3,11 @@ name: investigation-cost-guardrail description: Cost guardrail for AWS DevOps Agent that covers ALL AWS services and native agent tools. Before the agent makes any paid API call, this skill estimates cost, enforces budgets per investigation, detects expensive operations across all services (Athena queries, S3 scans, DynamoDB scans, SageMaker inference, PromQL, etc.), enforces time window requirements, monitors cumulative call volume, and cancels if thresholds are exceeded. This skill applies to ALL investigations regardless of which services are involved. metadata: author: tqquresh, inesttia - version: "2.0.0" + version: "2.1.0" aws-devops-agent-skills.agent-types: "Incident RCA" aws-devops-agent-skills.aws-services: "All" aws-devops-agent-skills.technical-domains: "Cost Optimization, Operations" --- - # Investigation Cost Guardrail Skill ## Overview @@ -27,6 +26,14 @@ Rather than listing every free/paid operation across 200+ AWS services, this ski This skill MUST be ALWAYS ACTIVE during investigations. It does NOT require user invocation. +## Fetch Live Rate Before Estimating + +The **first time** an operation is classified **PAID** by Layer 2, fetch the live rate **before** estimating cost. + +For **AWS operations**: read `references/pricing-reference.md` for the exact Pricing API call patterns, filter fields, filter values, and failure handling. The reference file specifies — for each service and operation — whether to use `Field:"operation"` or `Field:"usagetype"`, and the exact value to use. Do not derive these from the operation name. + +For **non-AWS tools** (Splunk, Datadog, Grafana, etc.): use the cost model from Layer 0 directly — no live lookup available. + --- ## Layer 0: Native Agent Tool Classification @@ -37,7 +44,7 @@ Before Layer 1 heuristics, classify the agent's own tools. These are NOT `use_aw | Tool | Classification | Cost Model | Guardrail | |---|---|---|---| -| `get_prometheus_metrics` | **PAID** | $0.01 / 1,000 metrics×periods (same billing meter as CloudWatch GetMetricData) | Track series × datapoints per call | +| `get_prometheus_metrics` | **PAID** | Billed per sample scanned — **rate from live `CW:PromQL:SamplesScanned` usagetype lookup** (`AmazonCloudWatch`, workload-region prefix) | Track samples scanned per call; HALT if rate lookup returns 0 products | | `use_aws` | **VARIABLE** | Depends on operation — apply Layers 1–3 | Full heuristic pipeline | | `use_azure` | **FREE** | Azure Reader role, no per-call billing | Track count only | | `grafana_query_prometheus` | **CAUTION** | Depends on Grafana data source billing model | Track count, warn at 50+ | @@ -55,15 +62,18 @@ Before Layer 1 heuristics, classify the agent's own tools. These are NOT `use_aw `get_prometheus_metrics` deserves special handling because: -- Cost = (number of series returned) × (number of datapoints per series) / 1000 × $0.01 -- Maximum 500 series per query — a broad query hitting the cap costs ~$0.005 per period -- Range queries with small `step` multiply cost: `7d / 60s step = 10,080 datapoints × 500 series = 5M metrics` +- Maximum 500 series per query — a broad query hitting the cap costs `500 × rate` per period. +- Range queries with small `step` multiply cost: `7d / 60s step = 10,080 datapoints × 500 series = 5M samples` **Before each PromQL call:** ```text -estimated_metrics = min(500, estimated_series) × (time_range_seconds / step_seconds) -estimated_cost = (estimated_metrics / 1000) * 0.01 +rate = live usagetype lookup (AmazonCloudWatch, usagetype=-CW:PromQL:SamplesScanned) + # bare "CW:PromQL:SamplesScanned" for us-east-1; use workload-region prefix for all others +if rate lookup returns 0 products: 🚫 HALT — do not hardcode or improvise a rate + +estimated_samples = min(500, estimated_series) × (time_range_seconds / step_seconds) +estimated_cost = estimated_samples × rate if estimated_cost > $0.50: ⚠️ WARN — suggest narrower time range, larger step, or label filters @@ -92,9 +102,7 @@ An operation is FREE if it matches ALL of these: - It returns metadata/configuration (not data content or query results) - It does NOT scan, process, or transform customer data -**Examples:** `DescribeInstances`, `ListFunctions`, `GetRole`, `LookupEvents` - -> ⚠️ **Exception:** Some services charge per-request even for Get/List operations. Layer 2 overrides this heuristic for: **S3** ($0.0004/1K GET, $0.005/1K LIST), **SQS** ($0.40/1M requests after free tier), **Lambda Invoke** ($0.20/1M). When Layer 2 has an entry, it takes precedence over Rule 1. +⚠️ Exception: Some services charge per-request even for Get/List operations. Layer 2 overrides this heuristic for S3 and Lambda Invoke — when Layer 2 has an entry, it takes precedence over Rule 1. > ⚠️ **Tool policy can override cost classification.** Some operations classified as FREE here (e.g., `cloudtrail:LookupEvents`) may be blocked by tool policy in certain environments. If an operation is denied, it costs $0.00 (never executed) — proceed with alternatives. @@ -104,13 +112,13 @@ An operation is PAID if it matches ANY of these patterns: | Pattern | Why It Costs Money | Examples | |---|---|---| -| Verb contains `Query` or `Search` | Scans indexed data | `StartQuery`, `Search`, `StartQueryExecution` | +| Verb contains `Query` | Scans indexed data | `StartQuery`, `StartQueryExecution` | | Verb contains `Scan` | Full table/index scan | `Scan` (DynamoDB), `StartScan` | | Verb contains `Execute` + processes data | Runs a computation | `StartQueryExecution` (Athena), `ExecuteStatement` | | Verb contains `Invoke` + runs workload | Triggers compute | `InvokeEndpoint` (SageMaker), `Invoke` (Lambda) | | Operation reads **content** (not metadata) | Data transfer | `GetObject` (S3, large), `GetLogEvents` (bulk), `BatchGetTraces` | -| Operation starts a **streaming session** | Per-time billing | `StartLiveTail`, `SubscribeToShard` | -| Operation name contains `Insights` | Analytics processing | `GetContributorInsights`, `StartQuery` | +| Operation starts a **streaming session** | Per-time billing | `StartLiveTail` | +| Operation name contains `Insights` | Analytics processing | `GetContributorInsights`, `GetInsightRuleReport` | ### Rule 3: CAUTION — High-volume free operations @@ -133,33 +141,46 @@ If an operation doesn't clearly fit Rules 1–3: ## Layer 2: Known-Paid Registry -These operations have **confirmed pricing** with estimation formulas. This list is extensible — operators can add entries. +These operations have **confirmed pricing**. Before estimating, fetch the live rate via the Pricing API using the exact filter field and value from the table below — see `references/pricing-reference.md` for the bash call patterns and region prefix mapping. + +> **Critical lookup rule:** `usagetype` and `operation` are different Pricing API filter fields. The correct field and value for each operation are specified explicitly below — do NOT derive them from the operation name. + +### Pricing Lookup Rules + +```text +if len(products) == 0: + 🚫 HALT — Pricing lookup returned no results for : + Reason: filter field/value or workload-region prefix may be incorrect + Do NOT proceed with the paid operation. + Do NOT improvise a rate from memory, training data, or any other source. + Options: + → Re-check pricing-reference.md for the correct filter field, value, and region prefix + → Skip this operation and use a free alternative + → Report the lookup gap to the user +``` ### Confirmed Paid Operations -| Service | Operation | Cost Formula | Estimation Method | -|---|---|---|---| -| CloudWatch Logs | `StartQuery` | $0.005/GB scanned | Query `IncomingBytes` metric for time window | -| CloudWatch Logs | `StartLiveTail` | $0.01/minute | Duration-based | -| CloudWatch | `GetMetricData` | $0.01/1,000 metrics×periods | Count metrics and periods | -| CloudWatch | `get_prometheus_metrics` | $0.01/1,000 metrics×periods | Count series × (range/step) | -| X-Ray | `GetTraceSummaries` | $0.50/1M traces | Paginate or sample to estimate count | -| X-Ray | `BatchGetTraces` | $0.50/1M traces | Count trace IDs in request | -| Athena | `StartQueryExecution` | $5.00/TB scanned | Check table metadata; require `WHERE` clause | -| DynamoDB | `Scan` | RCU consumed (~$0.25/1M RCU) | Check `TableSizeBytes`; BLOCK unless user approves | -| DynamoDB | `Query` (broad) | RCU consumed | Check `ItemCount`; warn if > 10K items | -| S3 | `GetObject` | $0.0004/1,000 requests + $0.09/GB transfer | Count requests; flag if cross-region or >100MB | -| S3 | `ListObjectsV2`, `ListObjects` | $0.005/1,000 requests | Count calls; warn if paginating heavily | -| S3 | `PutObject`, `CopyObject` | $0.005/1,000 requests | Count calls | -| S3 | `SelectObjectContent` | $0.002/GB scanned + $0.0007/GB returned | Check object size | -| Resource Explorer | `Search` | $0.01/query (after 1000 free/month) | Count calls | -| SageMaker | `InvokeEndpoint` | Instance-dependent | BLOCK — require explicit approval | -| Kinesis | `GetRecords` | $0.015/1M records | Estimate from shard count × duration | -| CloudWatch | Contributor Insights | $0.02/rule/1K events | Count rules and event volume | -| SQS | All operations | $0.40/1M requests (first 1M free/month) | Count total SQS calls; usually negligible | -| Lambda | `Invoke` | $0.20/1M requests + compute ($0.0000166667/GB-sec) | BLOCK unless user explicitly requests function execution | - -> ℹ️ **Rates are baseline published figures and may vary by region.** The regional rate may be higher, so a rate-based estimate is a lower bound. Treat any estimate within 20% of the remaining budget as exceeding it. +> **Rate = `pricePerUnit.USD`** from `terms.OnDemand → priceDimensions` where `beginRange="0"`. No /1K or /1M divisors. Region scoping: see Region Scoping column. + +| ServiceCode | Layer 2 Operation | Pricing API Filter Field | Filter Value | Region scoping | Cost Formula | Estimation Method | +|---|---|---|---|---|---|---| +| `AmazonCloudWatch` | `GetMetricData` | `operation` | `GetMetricData` | + `regionCode=` | `(metrics × periods) × rate` | Count metrics and periods | +| `AmazonCloudWatch` | `StartQuery` | `operation` | `StartQuery` | + `regionCode=` | `scan_gb × rate` | Query `IncomingBytes` metric for time window | +| `AmazonCloudWatch` | `StartLiveTail` | `operation` | `StartLiveTail` | + `regionCode=` | Duration-based | Duration-based | +| `AmazonCloudWatch` | `GetInsightRuleReport` | `usagetype` | `CW:GIRR-Metrics` | workload-region prefix required (bare in us-east-1) | metrics_requested × rate| Count metrics requested in the report call | +| `AmazonCloudWatch` | `get_prometheus_metrics` (native tool) | `usagetype` | `CW:PromQL:SamplesScanned` | workload-region prefix required (bare in us-east-1) | `samples_scanned × rate` | Estimate `min(500, series) × (range_seconds / step_seconds)`| +| `AWSXRay` | `GetTraceSummaries` | `operation` | `XRay-Traces-Scanned` | + `regionCode=` | `traces × rate` | Paginate or sample to estimate count | +| `AWSXRay` | `BatchGetTraces` | `operation` | `XRay-Traces-Retrieved` | + `regionCode=` | `traces × rate` | Count trace IDs in request | +| `AmazonAthena` | `StartQueryExecution` | `usagetype` | `DataScannedInTB` | workload-region prefix required (USE1- for us-east-1) | `scan_tb × rate`; min 10MB | Check table metadata; require `WHERE` clause | +| `AmazonDynamoDB` | `Scan` | `usagetype` | `ReadRequestUnits` | workload-region prefix required (**bare in us-east-1**, no USE1-) | RCU consumed × rate | Check `TableSizeBytes`; BLOCK unless user approves | +| `AmazonDynamoDB` | `Query` | `usagetype` | `ReadRequestUnits` | workload-region prefix required (**bare in us-east-1**, no USE1-) | RCU consumed × rate | Check `ItemCount`; warn if > 10K items | +| `AmazonS3` | `GetObject` | `usagetype` | `Requests-Tier2` | workload-region prefix required (bare in us-east-1) | See pricing-reference.md | Count requests; flag if cross-region or >100MB | +| `AmazonS3` | `ListObjectsV2`, `ListObjects` | `usagetype` | `Requests-Tier1` | workload-region prefix required (bare in us-east-1) | See pricing-reference.md | Count calls; warn if paginating heavily | +| `AmazonS3` | `PutObject`, `CopyObject` | `usagetype` | `Requests-Tier1` | workload-region prefix required (bare in us-east-1) | See pricing-reference.md | Count calls | +| `AmazonS3` | `SelectObjectContent` | `usagetype` | Bills on 3 meters — see pricing-reference.md | workload-region prefix required (bare in us-east-1) | See pricing-reference.md | Check object size | +| `AmazonSageMaker` | `InvokeEndpoint` | — | — | — | — | BLOCK — require explicit user approval | +| `AWSLambda` | `Invoke` | — | — | — | Per request + compute | BLOCK unless user explicitly requests function execution | --- @@ -173,9 +194,9 @@ After ANY operation executes, check the response for metered fields: |---|---|---| | `BytesScanned`, `DataScanned` | Data scanning charge | Record GB scanned, add to running cost | | `RecordsProcessed`, `ItemCount` | Record processing | Record count, estimate RCU/cost | -| `QueryExecutionId` + `DataScannedInBytes` | Athena scan | Add to cost at $5/TB | -| `TracesProcessedCount` | X-Ray processing | Add to cost at $0.50/1M | -| `ConsumedCapacity` | DynamoDB RCU/WCU | Add to cost at $0.25/1M RCU | +| `QueryExecutionId` + `DataScannedInBytes` | Athena scan | Add to cost at live rate | +| `TracesProcessedCount` | X-Ray processing | Add to cost at live rate | +| `ConsumedCapacity` | DynamoDB RCU/WCU | Add to cost at live rate | | `ContentLength` > 100MB | Large object fetch | Flag for transfer cost | | `NextToken` after 10+ pages | Pagination runaway | Trigger volume guardrail | | `warnings` containing "500 series" | PromQL truncation | Flag max-cost query, suggest narrowing | @@ -184,7 +205,7 @@ If a previously-unclassified operation returns metered fields: 1. Log it as a paid operation for this session 2. Add the cost to the running total -3. Warn the user: `⚠️ Discovered paid operation: : cost $X.XX` +3. Warn the user: `⚠️ Discovered paid operation: : cost $X.XX` --- @@ -212,14 +233,14 @@ else: proceed # After execution: running_cost += actual_cost (from response fields or estimation) - call_counts[service] += 1 + call_counts[servicecode] += 1 ``` **Volume guardrails:** ```text -if call_counts[any_service] > 200: ⚠️ WARN -if call_counts[any_service] > 500: 🚫 HALT +if call_counts[any_servicecode] > 200: ⚠️ WARN +if call_counts[any_servicecode] > 500: 🚫 HALT if sum(all_call_counts) > 1000: 🚫 HALT ``` @@ -229,12 +250,12 @@ if sum(all_call_counts) > 1000: 🚫 HALT ```text 📋 INVESTIGATION BUDGET STATUS -═══════════════════════════════════════════════════════════ +════════════════════════════════════════════════════════════ Budget: $10.00 Spent: $X.XX (Y paid operations) Free calls: Z operations (no cost) -PromQL: X,XXX metrics×periods consumed ($X.XX) -Next op: : — estimated $X.XX +PromQL: X,XXX samples scanned ($X.XX) +Next op: : — estimated $X.XX Projected: $X.XX (exceeds budget by $X.XX) 🚫 HALTED — would exceed $10.00 budget. @@ -269,8 +290,11 @@ For EVERY paid operation: ```text if target_region ≠ agent_space_region: + fetch transfer_rate = transfer_rate_cache[target_region] + ?? live lookup (see references/pricing-reference.md) + # If the live lookup returns 0 products: 🚫 HALT — do NOT improvise a rate estimated_return_size = estimate_return_bytes(operation_type) - transfer_cost = estimated_return_size × $0.02/GB + transfer_cost = estimated_return_size × transfer_rate total_estimate += transfer_cost flag: "⚠️ Cross-region transfer: → " ``` @@ -315,5 +339,3 @@ When halting or warning, ALWAYS suggest free or cheaper alternatives: | `athena:StartQueryExecution` (full) | Add partition filter in `WHERE` | 90%+ | | `xray:GetTraceSummaries` (broad) | Narrow time + add filter expression | 90%+ | | `s3:GetObject` (large) | `s3:SelectObjectContent` with SQL | Variable | - ---- diff --git a/skills/investigation-cost-guardrail/references/pricing-reference.md b/skills/investigation-cost-guardrail/references/pricing-reference.md new file mode 100644 index 0000000..a576509 --- /dev/null +++ b/skills/investigation-cost-guardrail/references/pricing-reference.md @@ -0,0 +1,162 @@ +# AWS Pricing Reference + +This file contains all AWS-specific Pricing API call patterns. Read this file the first time any operation is classified PAID by Layer 2, before estimating cost. + +> **Critical:** `usagetype` and `operation` are **different** Pricing API filter fields. Always use the field and value specified in the Layer 2 table — do NOT derive the filter value from the AWS API operation name. Every rate must come from an exact Pricing API lookup with the correct filter field/value and the correct region scoping. If any of these cannot be resolved exactly, **HALT** — do NOT improvise a rate. + +--- + +## Two different regions — do not conflate + +1. **Pricing API endpoint region** — always `us-east-1`. +2. **Workload region** — the region of the resource being priced. For `operation`-based lookups this is passed as a `regionCode` filter. For `usagetype`-based lookups this drives the prefix (`USW2-`, `EU-`, etc.). + +> The workload region is a function of the **resource being priced only** — never the agent's runtime region, and never a hardcoded default. If the workload region is unknown, resolve it from the resource ARN before pricing. **Do NOT default to us-east-1** as the workload region. + +--- + +## Region Scoping by Filter Type + +| Filter type | How to scope to workload region | +|---|---| +| `operation` | Add `{"Type":"TERM_MATCH","Field":"regionCode","Value":""}` as a second filter. Returns exactly 1 product. Omitting it returns all regions (paginated, nondeterministic). | +| `usagetype` | Prepend the workload-region prefix to the value (e.g. `USW2-DataScannedInTB`). See Region Prefix Mapping table below. | + +--- + +## Standard Patterns + +### operation-based lookup + +```bash +aws pricing get-products \ + --service-code \ + --filters '[{"Type":"TERM_MATCH","Field":"operation","Value":""}, + {"Type":"TERM_MATCH","Field":"regionCode","Value":""}]' \ + --region us-east-1 +``` + +### usagetype-based lookup + +```bash +aws pricing get-products \ + --service-code \ + --filters '[{"Type":"TERM_MATCH","Field":"usagetype","Value":"-"}]' \ + --region us-east-1 +``` + +### Empty result = lookup failure = HALT + +If the Pricing API returns zero products, **do not proceed**. This is a lookup failure, not a zero-cost result. + +```text +if len(products) == 0: + 🚫 HALT — Pricing lookup returned no results + Do NOT proceed with the paid operation. + Do NOT improvise a rate from memory, training data, or any other source. +``` + +> Every operation that returns empty HALTs. There are no exceptions. + +> ⚠️ **X-Ray eu-west-3 catalog gap:** `XRay-Traces-Scanned` and `XRay-Traces-Retrieved` do not exist in the AWS Pricing catalog for eu-west-3. This is expected — HALT-on-empty applies, report the gap to the user. + +--- + +## Amazon S3 — usagetype with region prefix + +```bash +aws pricing get-products \ + --service-code AmazonS3 \ + --filters '[{"Type":"TERM_MATCH","Field":"usagetype","Value":"-Requests-"}]' \ + --region us-east-1 +``` + +**us-east-1 uses a bare value (no prefix):** `Requests-Tier2`. All other regions carry the prefix: `USW2-Requests-Tier2`. If the lookup returns 0 products, HALT. + +#### S3 Tier Mapping + +| Tier | Operations | +|---|---| +| `Tier1` | PUT, COPY, POST, LIST | +| `Tier2` | GET, SELECT, HEAD | + +#### S3 Select (`SelectObjectContent`) — three meters + +The request tier prices the **call**, not the bytes. Select needs all three lookups below; `usagetype` is the only viable filter, as these carry an empty `operation` field. + +| Component | usagetype | Unit | Formula | +|---|---|---|---| +| Bytes scanned | `-Select-Scanned-Bytes` | GB | `scan_gb × rate` | +| Bytes returned | `-Select-Returned-Bytes` | GB | `returned_gb × rate` | +| Request | `-Requests-Tier2` | Requests | `requests × rate` | + +`total = scanned + returned + request`. Bare values in us-east-1, prefixed elsewhere. If any lookup returns 0 products, HALT. + +#### Region Prefix Mapping (usagetype-based services: S3, Athena, DynamoDB, PromQL) + +| Region | Prefix | +|---|---| +| us-east-1 | *(S3/DynamoDB/PromQL: omit — bare value; Athena: USE1-)* | +| us-east-2 | USE2 | +| us-west-1 | USW1 | +| us-west-2 | USW2 | +| eu-west-1 | EU | +| eu-west-2 | EUW2 | +| eu-west-3 | EUW3 | +| eu-central-1 | EUC1 | +| eu-central-2 | EUC2 | +| eu-north-1 | EUN1 | +| eu-south-1 | EUS1 | +| eu-south-2 | EUS2 | +| ap-southeast-1 | APS1 | +| ap-southeast-2 | APS2 | +| ap-southeast-3 | APS4 | +| ap-southeast-4 | APS6 | +| ap-southeast-5 | APS7 | +| ap-southeast-6 | APS8 | +| ap-southeast-7 | APS9 | +| ap-northeast-1 | APN1 | +| ap-northeast-2 | APN2 | +| ap-northeast-3 | APN3 | +| ap-south-1 | APS3 | +| ap-south-2 | APS5 | +| ap-east-1 | APE1 | +| ap-east-2 | APE2 | +| sa-east-1 | SAE1 | +| ca-central-1 | CAN1 | +| ca-west-1 | CAN2 | +| me-south-1 | MES1 | +| me-central-1 | MEC1 | +| mx-central-1 | MXC1 | +| af-south-1 | AFS1 | +| il-central-1 | ILC1 | + +--- + +## Cross-Region Data Transfer Rate + +For any paid operation where the source (workload) region differs from the destination (agent space) region, fetch the live transfer rate. Supplying **both** `fromRegionCode` and `toRegionCode` (plus `transferType` and `toLocationType`) narrows the result to a single deterministic product; a `fromRegionCode`-only filter returns many products and must not be used. + +```bash +aws pricing get-products \ + --service-code AWSDataTransfer \ + --filters '[{"Type":"TERM_MATCH","Field":"transferType","Value":"InterRegion Outbound"}, + {"Type":"TERM_MATCH","Field":"fromRegionCode","Value":""}, + {"Type":"TERM_MATCH","Field":"toRegionCode","Value":""}, + {"Type":"TERM_MATCH","Field":"toLocationType","Value":"AWS Region"}]' \ + --region us-east-1 +``` + +- ``: region where data originates (the workload region) — resolve from the resource ARN; do NOT default it +- ``: the agent space region — resolve it explicitly; do NOT hardcode us-east-1 +- Select the price dimension with `beginRange: "0"` if multiple are returned +- Cache as `transfer_rate_cache[source_region → destination_region]` — one lookup per route per investigation +- **If the filtered query returns 0 results: 🚫 HALT.** Do NOT improvise a rate. Re-check the region codes and filters, or report the gap. + +> Inter-region transfer rates vary widely by geography (roughly $0.01–$0.15/GB depending on source region) — always look up the specific route; never assume a flat rate. + +--- + +## Reference Links + +[CloudWatch](https://aws.amazon.com/cloudwatch/pricing/) · [X-Ray](https://aws.amazon.com/xray/pricing/) · [Athena](https://aws.amazon.com/athena/pricing/) · [DynamoDB](https://aws.amazon.com/dynamodb/pricing/on-demand/) · [S3](https://aws.amazon.com/s3/pricing/) · [Lambda](https://aws.amazon.com/lambda/pricing/) · [Data Transfer](https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer)