Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions cli/azd/cmd/telemetry_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,21 @@ func TestTelemetryFieldConstants(t *testing.T) {
require.Equal(t, int64(3), kvCount.Value.AsInt64())
})

t.Run("GDPRMeasurementMetadata", func(t *testing.T) {
t.Parallel()

measurementFields := []fields.AttributeKey{
fields.AgentFixAttempts,
fields.ExeGraphMaxConcurrencyKey,
fields.ToolExitCode,
}
for _, field := range measurementFields {
require.True(t, field.IsMeasurement, field.Key)
}

require.False(t, fields.ServiceErrorCode.IsMeasurement)
})

// Hooks command telemetry fields
t.Run("HooksFields", func(t *testing.T) {
t.Parallel()
Expand Down
8 changes: 5 additions & 3 deletions cli/azd/internal/tracing/fields/fields.go
Original file line number Diff line number Diff line change
Expand Up @@ -799,6 +799,7 @@ var (
Key: attribute.Key("exegraph.max_concurrency"),
Classification: SystemMetadata,
Purpose: PerformanceAndHealth,
IsMeasurement: true,
}

// ExeGraphErrorPolicyKey records the error policy (fail_fast or continue_on_error).
Expand Down Expand Up @@ -945,8 +946,8 @@ var (
Purpose: PerformanceAndHealth,
}

// Status code of a response returned by the service.
// For HTTP, this corresponds to the HTTP status code.
// Status code of a response returned by the service. Numeric HTTP/service
// statuses are measurements; AAD authentication errors use string OAuth statuses.
ServiceStatusCode = AttributeKey{
Key: attribute.Key("service.statusCode"),
Classification: SystemMetadata,
Expand All @@ -968,7 +969,6 @@ var (
Key: attribute.Key("service.errorCode"),
Classification: SystemMetadata,
Purpose: PerformanceAndHealth,
IsMeasurement: true,
}

// Correlation ID for a request to the service.
Expand All @@ -993,6 +993,7 @@ var (
Key: attribute.Key("tool.exitCode"),
Classification: SystemMetadata,
Purpose: PerformanceAndHealth,
IsMeasurement: true,
}
)

Expand Down Expand Up @@ -1198,6 +1199,7 @@ var (
Key: attribute.Key("agent.fix.attempts"),
Classification: SystemMetadata,
Purpose: FeatureInsight,
IsMeasurement: true,
}
)

Expand Down
21 changes: 12 additions & 9 deletions docs/reference/telemetry-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,14 +268,17 @@ The `ResultCode` field classifies errors into categories. Understanding this tax

### Service Attributes (Azure API Calls)

These attributes are emitted as classified error details. `MapError` prefixes their declared
`service.*` keys with `error.`, so the table lists the runtime keys used in queries.

| Field Key | Type | Description |
|-----------|------|-------------|
| `service.host` | string | Azure service host |
| `service.name` | string | Azure service name (on service call spans) |
| `service.statusCode` | measurement | HTTP status code |
| `service.method` | string | HTTP method |
| `service.errorCode` | measurement | Service-specific error code |
| `service.correlationId` | string | Azure correlation ID |
| `error.service.host` | string | Azure service host |
Comment thread
hemarina marked this conversation as resolved.
| `error.service.name` | string | Azure service name associated with the failure |
| `error.service.statusCode` | measurement or string | Numeric HTTP/service status code; AAD authentication errors use a string OAuth status such as `invalid_grant` |
| `error.service.method` | string | HTTP method |
| `error.service.errorCode` | string | Service-specific error code; some ARM deployment errors encode structured JSON |
| `error.service.correlationId` | string | Azure correlation ID |

### Tool Invocation Attributes (External CLI Tools)

Expand Down Expand Up @@ -574,7 +577,7 @@ The first-run middleware is not currently registered, so these fields are not em
| Field Key | Type | Description |
|-----------|------|-------------|
| `exegraph.step.count` | measurement | Total steps in graph |
| `exegraph.max_concurrency` | string | Effective concurrency limit |
| `exegraph.max_concurrency` | measurement | Effective concurrency limit |
| `exegraph.error_policy` | string | `fail_fast` or `continue_on_error` |
| `exegraph.step.name` | string | Step name. **SHA-256 hashed** — embeds user-defined service/layer names from `azure.yaml` |
| `exegraph.step.deps` | string[] | Step dependencies (other step names). **SHA-256 hashed** for the same reason |
Expand Down Expand Up @@ -618,7 +621,7 @@ The first-run middleware is not currently registered, so these fields are not em

| Field Key | Type | Description |
|-----------|------|-------------|
| `agent.fix.attempts` | string | Number of fix attempts |
| `agent.fix.attempts` | measurement | Number of fix attempts |
</details>

### Execution Environments
Expand Down Expand Up @@ -767,7 +770,7 @@ Many failed commands produce the catch-all result code `internal.errors_errorStr

**To investigate these errors:**
1. Check `error.chain.types` (if available) for the full error type chain
2. Correlate with `service.errorCode` or `service.statusCode` for Azure API failures
2. Correlate with `error.service.errorCode` or `error.service.statusCode` for Azure API failures
3. Look at surrounding span context (same `OperationId`) for additional detail

### Hashed Fields and Template Joins
Expand Down
2 changes: 1 addition & 1 deletion docs/specs/metrics-audit/feature-telemetry-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ reserved field contracts.
| **Provision validation** | `provision` (all providers, plus Bicep `arm-provision` prior to ARM deploy) | `validation.provision` | `validation.provision.outcome`, plus peer fields covering warnings/errors counts, cancel reason, and `check_type` (dispatch site) | Local-only validation; runs for every provider via the provider-agnostic `provision` dispatch and additionally as Bicep `arm-provision`. `check_type` distinguishes the two emissions so Bicep provisions are not double-counted |
| **ARM deployment client** | `provision` (any Bicep flow) | `arm.deploy.subscription`, `arm.deploy.resourcegroup`, `arm.stack.deploy.subscription`, `arm.stack.deploy.resourcegroup`, `arm.whatif.subscription`, `arm.whatif.resourcegroup`, `arm.validate.subscription`, `arm.validate.resourcegroup` | ARM operation status + duration | Per-call instrumentation in the ARM client; covers regular + stack deployments at both scopes |
| **Multi-layer provision** | `provision` (when `infra.layers[]` is configured in `azure.yaml`) | (none — enriches the `provision` span) | `provision.layer.count`, `provision.layer.max_parallel`, `provision.layer.safe_fallback_count`, `provision.layer.explicit_dependson_count` | All four are integer measurements emitted from `internal/cmd/provision_graph.go`; no per-layer duration or outcome attribute is emitted |
| **Execution graph (scheduler)** | `up`, `provision`, `deploy`, `package`, `publish`, `down` | `exegraph.run`, `exegraph.step` | `exegraph.step.count`, `exegraph.max_concurrency`, `exegraph.error_policy`, `exegraph.step.name` (hashed), `exegraph.step.deps` (hashed slice), `exegraph.step.tags` (raw — hardcoded literals only), `exegraph.step.timeout_s` | Step names embed user-defined service / layer names from `azure.yaml`; both `name` and `deps` use `fields.StringHashed` / `fields.StringSliceHashed` |
| **Execution graph (scheduler)** | `up`, `provision`, `deploy`, `package`, `publish`, `down` | `exegraph.run`, `exegraph.step` | `exegraph.step.count`, `exegraph.max_concurrency`, `exegraph.error_policy`, `exegraph.step.name` (hashed), `exegraph.step.deps` (hashed slice), `exegraph.step.tags` (raw — hardcoded literals only), `exegraph.step.timeout_s` | Step count, max concurrency, and timeout are integer measurements; step names embed user-defined service / layer names from `azure.yaml`; both `name` and `deps` use `fields.StringHashed` / `fields.StringSliceHashed` |
| **Container lifecycle** | `package`, `deploy` (container service targets) | `container.credentials`, `container.publish`, `container.remotebuild` | `container.publish` sets a `container.remotebuild` property (bool) only; the `container.credentials` and `container.remotebuild` events set no attributes (span status carries success/failure and duration) | The hashed `pack.builder.image` / `pack.builder.tag` attributes are emitted on the separate `tools.pack.build` span, not the `container.*` spans |
| **App Service deploy** | `deploy`, `publish` (App Service targets) | `deploy.appservice.zip` | `deploy.appservice.linux` (bool), `deploy.appservice.attempt` (retry attempt number) | Zip-deploy path only; outcome / duration are carried by the span status and span timing, not by dedicated attributes |
| **AKS service target** | `provision` (AKS preprovision/postprovision) | `aks.postprovision.skip` | `skip.reason` (bounded enum — `cluster_not_provisioned`) | Recorded when cluster is not yet available for context setup |
Expand Down
20 changes: 10 additions & 10 deletions docs/specs/metrics-audit/telemetry-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,17 +131,17 @@ not emitted by azd spans.
|-------|----------|----------------|---------|-------|
| Service host | `service.host` | SystemMetadata | PerformanceAndHealth | |
| Service name | `service.name` | SystemMetadata | PerformanceAndHealth | |
| Status code | `service.statusCode` | SystemMetadata | PerformanceAndHealth | **Measurement** |
| Status code | `service.statusCode` | SystemMetadata | PerformanceAndHealth | **Measurement or string** — numeric HTTP/service status codes are measurements; AAD authentication errors emit string OAuth statuses such as `invalid_grant` |
| Method | `service.method` | SystemMetadata | PerformanceAndHealth | |
| Error code | `service.errorCode` | SystemMetadata | PerformanceAndHealth | **Measurement**; ARM deployment errors encode JSON objects with `error.code` and `error.arm.frame_index` |
| Error code | `service.errorCode` | SystemMetadata | PerformanceAndHealth | String; ARM deployment errors encode JSON objects with `error.code` and `error.arm.frame_index` |
| Correlation ID | `service.correlationId` | SystemMetadata | PerformanceAndHealth | |

### Tool Attributes

| Field | OTel Key | Classification | Purpose |
|-------|----------|----------------|---------|
| Tool name | `tool.name` | SystemMetadata | FeatureInsight |
| Tool exit code | `tool.exitCode` | SystemMetadata | PerformanceAndHealth |
| Field | OTel Key | Classification | Purpose | Notes |
|-------|----------|----------------|---------|-------|
| Tool name | `tool.name` | SystemMetadata | PerformanceAndHealth | |
| Tool exit code | `tool.exitCode` | SystemMetadata | PerformanceAndHealth | **Measurement** |

### Performance

Expand Down Expand Up @@ -208,9 +208,9 @@ not emitted by azd spans.

### Agent

| Field | OTel Key | Classification | Purpose |
|-------|----------|----------------|---------|
| Fix attempts | `agent.fix.attempts` | SystemMetadata | PerformanceAndHealth |
| Field | OTel Key | Classification | Purpose | Notes |
|-------|----------|----------------|---------|-------|
| Fix attempts | `agent.fix.attempts` | SystemMetadata | FeatureInsight | **Measurement** |

### Extensions

Expand Down Expand Up @@ -415,7 +415,7 @@ The execution graph powers the parallel `up` / `provision` / `deploy` engine.
| Field | OTel Key | Classification | Purpose | Notes |
|-------|----------|----------------|---------|-------|
| Step count | `exegraph.step.count` | SystemMetadata | PerformanceAndHealth | **Measurement** — total number of steps in the graph |
| Max concurrency | `exegraph.max_concurrency` | SystemMetadata | PerformanceAndHealth | Effective concurrency limit used for the run |
| Max concurrency | `exegraph.max_concurrency` | SystemMetadata | PerformanceAndHealth | **Measurement** — effective concurrency limit used for the run |
| Error policy | `exegraph.error_policy` | SystemMetadata | PerformanceAndHealth | `fail_fast` or `continue_on_error` |
| Step name | `exegraph.step.name` | SystemMetadata | PerformanceAndHealth | **Hashed** via `fields.StringHashed` — step names embed user-chosen service / layer names from `azure.yaml` (e.g., `deploy-<svc.Name>`, `<layer.Name>`) |
| Step deps | `exegraph.step.deps` | SystemMetadata | PerformanceAndHealth | **Hashed slice** via `fields.StringSliceHashed` — each entry is another step name that embeds user-chosen identifiers |
Expand Down
Loading