Skip to content

Fix issue with duplicate prometheus HELP/TYPE lines - #772

Merged
cigamit merged 2 commits into
mainfrom
duplicate_metrics
Aug 31, 2026
Merged

Fix issue with duplicate prometheus HELP/TYPE lines#772
cigamit merged 2 commits into
mainfrom
duplicate_metrics

Conversation

@cigamit

@cigamit cigamit commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Resolves #766

@cigamit
cigamit requested a review from TheWitness August 31, 2026 04:06
@cigamit cigamit self-assigned this Aug 31, 2026
@cigamit cigamit added the bug Something isn't working label Aug 31, 2026
Copilot AI lite review requested due to automatic review settings August 31, 2026 04:06
@cigamit cigamit added the python Pull requests that update python code label Aug 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Pull request overview

Fixes invalid Prometheus text exposition for /api/v2/metrics by ensuring shared “operational” metrics are emitted with a single # HELP/# TYPE header across namespaces, while still including per-namespace samples via the subsystem label.

Changes:

  • Refactors metric rendering to separate header emission from sample emission (include_header, prometheus_header) and aggregates operational metric samples across namespaces.
  • Adds OPERATIONAL_FIELDS to centrally define which metrics must be grouped/deduped.
  • Adds a unit test to verify Prometheus parsing and prevent duplicate HELP/TYPE output regressions.
File summaries
File Description
awx/main/analytics/subsystem_metrics.py Aggregates operational metric samples and emits a single header per metric to avoid duplicate HELP/TYPE lines.
awx/main/tests/unit/analytics/test_subsystem_metrics.py Adds a regression test ensuring /api/v2/metrics output is parseable and does not duplicate declarations.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread awx/main/tests/unit/analytics/test_subsystem_metrics.py Outdated
Copilot AI review requested due to automatic review settings August 31, 2026 04:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

awx/main/analytics/subsystem_metrics.py:186

  • HistogramM.to_prometheus() assumes instance_data[instance][self.field] is always present. In mixed-version clusters or after upgrades, stale instances can lack newer metrics; BaseM.to_prometheus() already guards against this, but HistogramM will raise a KeyError and break the entire /api/v2/metrics response.
        for instance in instance_data:
            # Build label string
            node_label = f'node="{instance}"'
            subsystem_label = f',subsystem="{namespace}"' if namespace else ''
            for i, b in enumerate(self.buckets):
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@cigamit
cigamit merged commit 121174d into main Aug 31, 2026
10 checks passed
@cigamit
cigamit deleted the duplicate_metrics branch August 31, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working python Pull requests that update python code

Development

Successfully merging this pull request may close these issues.

api/v2/metrics is invalid prometheus format: duplicate HELP and TYPE lines

3 participants