Skip to content

[Metrics] Query latency histogram #1409

Description

@alexkarp-umd

PgDog exposes query latency as total_query_time and avg_query_time. Both are means over a pool. As an on-call SRE I want to have percentiles available since averages alone tell you very little about worst-case performance about the distribution of request latencies.

I propose adding a per-pool histogram of individual query durations to OpenMetrics and OTLP, so histogram_quantile() works against PgDog's metrics:

histogram_quantile(0.99, sum by (le, database) (rate(pgdog_query_time_seconds_bucket[5m])))

Proposal

  • OpenMetrics: _bucket / _sum / _count, cumulative counts, trailing le="+Inf".
  • OTLP: a real Histogram data point with explicitBounds / bucketCounts and support for both cumulative and delta mode (see [Metrics] Datapoints follow OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE #1293)
  • Default ladder 100µs → 30s in 12 bounds, plus the implicit +Inf, so 13 series per pool.
  • Bucket count and boundaries configurable at startup.

Status

I have this working on a branch rebased onto #1293 and I can open a PR for it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions