Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Skills can be used with these AWS DevOps Agent types:
- [VPC DNS Investigation Skill](skills/aws-vpc-dns-investigation/SKILL.md): Diagnoses VPC DNS resolution failures and validates DNS control-plane changes before they are applied, driving the aws-vpc-dns-diagnostics MCP server to observe live resolution from inside the affected subnet and to simulate a proposed change
- [Bedrock Adoption Readiness Skill](skills/bedrock-adoption-readiness/SKILL.md): Assesses an AWS account's readiness to run Amazon Bedrock at production scale across IAM governance, data retention (ZDR), quota and capacity headroom, and operational observability, covering both the standard Bedrock and bedrock-mantle (OpenAI-compatible) surfaces with multi-region discovery
- [Analytics OpenSearch Expertise Skill](skills/analytics-opensearch-expertise/SKILL.md): Performs read-only health assessments of Amazon OpenSearch Service domains through 24 deterministic checks across cluster health, storage and shards, performance, security, and cost optimization, producing a structured findings report with prioritized remediation guidance
- [Analytics DataOps Expertise Skill](skills/analytics-dataops-expertise/SKILL.md): Performs read-only DataOps maturity assessments of an AWS data platform through 26 questions scored 1-5 across five dimensions (architecture, security and governance, incident management and observability, automation and testing, and cost), producing a structured scorecard with per-dimension roll-ups and prioritized remediation guidance

## Key Concepts

Expand Down
3 changes: 3 additions & 0 deletions skills/analytics-dataops-expertise/.skilleval.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
audit:
ignore:
- STR-016 # README alongside SKILL.md is intentional
42 changes: 42 additions & 0 deletions skills/analytics-dataops-expertise/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Changelog

All notable changes to the `analytics-dataops-expertise` skill are documented here.

## [1.0.2] - 2026-08-28
### Fixed
- Dimension fidelity: pinned the scorecard to EXACTLY five dimensions (Architecture;
Security & Governance; Incident Management & Observability; Automation & Testing; Cost)
with fixed per-question membership, and aligned the frontmatter description to those five.
Fixes an observed run where the agent reported "7 dimensions" by promoting Architecture
questions (Real-time Processing Q4, Resilience Q7/Q8) into standalone top-level dimensions.

## [1.0.1] - 2026-08-28
### Fixed
- Report rendering: added a LITERAL-CONTENT RULE and an incremental RENDERING METHOD
to the Output Format so the agent writes every section and all 26 score-matrix rows
with real computed values. Fixes an observed failure in DevOps Agent where a saved
artifact contained placeholder labels ("full 3-tier content…", `{"q":"Q3"}` rows with
empty columns) instead of the actual report. Reinforced the same requirement in
Execution Flow step 6.

## [1.0.0] - 2026-08-27
### Added
- Initial release: 26 read-only control-plane maturity questions across five dimensions
(Architecture; Security & Governance; Incident Management & Observability;
Automation & Testing; Cost), each scored on a 1-5 maturity scale.
- Per-question checks that name the AWS API(s), the field(s) to read, and the signal-to-rating
mapping — 100% control-plane / API-driven with no data-plane access and no AWS-internal
data sources.
- Auto-score ceilings: questions that cannot be fully confirmed from control-plane signals
alone (Q6, Q8, Q11, Q13, Q14, Q19, Q20, Q23, Q24, Q26, Q30, Q37, Q38, Q39) are capped at 3;
the report states when a higher score requires conversational confirmation.
- Remediation Reference: a 26-entry verbatim dictionary (why-it-matters, resolution steps,
and verified official AWS documentation links) in `references/remediation-reference.md`,
loaded on demand via `read_skill_resource`. Every question scored ≤ 3 pulls its entry
verbatim; documentation links come only from the dictionary, eliminating URL hallucination.
- Structured scorecard output: per-section and overall averages, tiered findings, prioritized
recommendations, a raw-data reference, and a mandatory 26-row score matrix with a self-count
check.
- Error handling: missing services and region-unavailable APIs are treated as score-1 signals,
not blockers; a question is only SKIPPED when every API it needs is denied by IAM. Cost
Explorer must be enabled and is called in `us-east-1`.
62 changes: 62 additions & 0 deletions skills/analytics-dataops-expertise/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# analytics-dataops-expertise

A read-only **DataOps maturity assessment** skill for AWS DevOps Agent. Given an account ID and region, it runs control-plane API checks across the data platform, scores each dimension on a 1-5 maturity scale, and produces a structured scorecard with prioritized, doc-linked recommendations.

## What it does

Scores a customer's AWS data platform across five dimensions (26 maturity questions total):

1. **Architecture** — catalog/governance patterns, real-time processing, change data capture, capacity planning, fault tolerance & HA, disaster recovery
2. **Security & Governance** — metadata management, lineage, data classification, data protection (encryption/PITR), fine-grained access control
3. **Incident Management & Observability** — workload analytics, monitoring & alerting, application tracing, drift detection, SLA management, user experience
4. **Automation & Testing** — infrastructure pipelines (IaC/CI-CD), orchestration, version/patch management, data quality testing
5. **Cost** — resource tagging, chargeback/showback, storage lifecycle management, data-processing cost, unused-resource cleanup

Each question maps a set of read-only APIs to specific fields and a 1-5 rating rule. Questions that can't be fully confirmed from control-plane signals alone are **capped at 3** (existence of a resource ≠ mature use of it); the report states when a higher score needs conversational confirmation.

It is **100% control-plane / API-driven** — no data-plane access (no catalog/table/index queries), no external packages, and no AWS-internal data sources. That makes it compatible with DevOps Agent without a custom MCP server.

## Prerequisites

- **Account ID and region** for the assessment — the scorer is account + region scoped. If either is missing, the skill asks before running.
- **AWS resources:** one or more analytics/data services in the target account/region (Glue, Kinesis, MSK, DMS, DynamoDB, RDS, OpenSearch, Redshift, EMR, MWAA, etc.). Absence of a service is itself a valid low-maturity signal.
- **Cost Explorer** must be enabled and is called in `us-east-1` (used by the two cost questions Q37/Q39). If it's unavailable, those score from tagging signals with the gap noted.

**IAM permissions** the DevOps Agent execution role needs — all read-only. Most are covered by a ViewOnly/read-only managed policy; attach a supplemental policy for any gaps:

- Glue / Lake Formation: `glue:GetDatabases`, `glue:GetJobs`, `glue:GetCrawlers`, `glue:ListRegistries`, `glue:ListSchemas`, `glue:ListDataQualityRulesets`, `glue:ListDataQualityResults`, `glue:ListWorkflows`, `lakeformation:GetDataLakeSettings`, `lakeformation:ListPermissions`, `lakeformation:ListDataCellsFilter`
- Streaming: `kinesis:ListStreams`, `firehose:ListDeliveryStreams`, `kinesisanalyticsv2:ListApplications`, `kafka:ListClustersV2`
- Migration/CDC: `dms:DescribeReplicationTasks`, `dms:DescribeReplicationInstances`, `dms:DescribeEndpoints`, `dms:DescribeEventSubscriptions`
- Databases: `dynamodb:ListTables`, `dynamodb:DescribeTable`, `dynamodb:DescribeContinuousBackups`, `rds:DescribeDBInstances`, `rds:DescribeDBEngineVersions`, `redshift:DescribeClusters`, `es:DescribeDomain`, `es:ListDomainNames`
- Scaling: `application-autoscaling:DescribeScalableTargets`, `autoscaling:DescribeAutoScalingGroups`
- Resilience: `backup:ListBackupPlans`
- Observability: `cloudwatch:DescribeAlarms`, `cloudwatch:ListDashboards`, `cloudwatch:GetMetricData`, `sns:ListTopics`, `events:ListRules`, `xray:GetSamplingRules`, `xray:GetGroups`, `rum:ListAppMonitors`, `quicksight:ListDashboards`
- Automation: `cloudformation:ListStacks`, `codepipeline:ListPipelines`, `codecommit:ListRepositories`, `mwaa:ListEnvironments`, `states:ListStateMachines`, `ssm:DescribePatchBaselines`, `lambda:ListFunctions`
- Governance/Security: `macie2:GetMacieSession`, `macie2:ListClassificationJobs`, `kms:ListKeys`, `s3:ListAllMyBuckets`, `s3:GetBucketLifecycleConfiguration`, `iam:ListPolicies`, `config:DescribeConfigRules`, `config:DescribeComplianceByConfigRule`
- Cost: `resourcegroupstaggingapi:GetResources`, `ce:GetCostAndUsage`, `ce:GetTags`, `cost-optimization-hub:ListRecommendations`

## How to use it with DevOps Agent

1. Get this skill directory onto your machine (clone the repo, or use GitHub's **Code → Download ZIP**), then zip just this skill folder:
```bash
cd tools-for-devops-agent/skills
zip -r analytics-dataops-expertise.zip analytics-dataops-expertise
```
The zip must contain `SKILL.md` at the top of the `analytics-dataops-expertise/` folder. ZIP only, max 6 MB, no `scripts/` directory (this skill has none).
2. In the DevOps Agent Operator Web App, go to **Knowledge → Skills → Add skill → Upload skill** and select the zip. Choose the agent types that can use it (Generic applies to all; or target Chat / Incident RCA / etc.). Confirm it shows as active.
3. Prompt in natural language without naming the skill, and include the account ID and region, e.g.:
- "Assess the DataOps maturity of account `123456789012` in `eu-west-1`."
- "Review our data platform's architecture, governance, and cost posture. Account `123456789012`, region `us-east-1`."
- "How mature is our data pipeline observability and automation?"
4. Review the agent's reasoning trace to confirm the skill activated and the checks ran. Start a new chat after re-uploading a changed version so the latest skill loads.

If the agent does not invoke the skill, refine the `description` field in `SKILL.md` (see "Optimizing description" in the Agent Skills specification).

## Non-production disclaimer

> ⚠️ This skill is sample code, not intended for production use without additional review and testing. Users should validate in a non-production environment first.

## Maintainers

- prasadnu
- genealpe
Loading