Add a metrics endpoint - #123
Open
beetlebugorg wants to merge 16 commits into
Open
Conversation
dims_metrics_init creates it in post config, before the first fork, and sizes it from AP_MPMQ_HARD_LIMIT_DAEMONS. dims_metrics_child_init claims a process slot. The block holds the counters, the histograms, and one slot per child. The index maps turn a status, a response code, and an image format into a slot, and hold every value outside their list in the last one. Nothing reads the block yet.
The bucket search, the histogram totals, and the four index maps. A bound belongs to its own bucket, and a value above the last bound belongs to +Inf. stub_httpd.c gains ap_mpm_query, which sizes the process slots.
DimsMetricsEnabled is the switch and defaults to off. A location that sets the dims-metrics handler serves Prometheus text 0.0.4 when the directive is on, and httpd returns 404 when it is off. The writer prints a family per counter and per histogram, with cumulative _bucket lines, a _sum, and a _count. Every value is zero until the next commits record them. A bucket bound goes through snprintf. The httpd formatter prints .005 for 0.005 and rounds 1048576 to 1048580, and a dashboard matches le as a string. src/metrics_report.c holds the writer, so the in-process cases link the block and the index maps without the httpd response API.
dims_requests_total, dims_responses_total, dims_request_duration_seconds, and dims_requests_in_flight. dims_handler sets d->endpoint and calls dims_metrics_request_begin, which increments the gauge and registers a pool cleanup. The cleanup decrements the gauge and records the outcome, the response code, and the duration, so every return path in the handler records exactly once.
dims_source_fetch_duration_seconds, dims_source_bytes, dims_source_bytes_total, dims_origin_responses_total, and dims_source_format_total. The request record already holds the duration, the size, and the origin status, so the same pool cleanup records them. dims_fetch_remote_image keeps the format the wand read as a metrics slot, because the string belongs to ImageMagick. download_time counts milliseconds, so a fetch inside one millisecond reports zero. It belongs in the first bucket rather than nowhere.
dims_imagemagick_duration_seconds, dims_source_frames, dims_output_format_total, dims_output_bytes_total, and dims_output_bytes. dims_send_image keeps the format and the blob length on the request record, and dims_process_image keeps the frame count. The pool cleanup reports them with the rest.
Each child writes MagickGetResource for the four limited resources, and its resident and virtual size from /proc/self/statm, into the slot it claimed. The endpoint sums the live slots and reports the widest one, so two series cover a pool of any size. MagickGetResourceLimit reads the ceiling a worker runs under, so a dashboard divides use by limit. /proc/self/statm is Linux only. A platform without it reports no memory gauges rather than zero.
dims_httpd_workers by scoreboard state, the process count and its ceiling, the thread count, connections by state, and the MPM generation. The scoreboard is the server's own record, so the endpoint reads it at scrape time rather than through the metrics block. Ready workers against the ceiling times the thread count is the headroom an operator reads before raising DIMS_SERVER_LIMIT.
dims_stats_rec and its shared memory block go away. dims_status_handler sums the requests table for the four numbers it prints, so the two endpoints report the same counters. dims_init set success_count to 1 at startup, so /dims-status/ reported one success before it served a request. Summing the table reports zero. The four apr_atomic_inc32 calls in mod_dims.c sat in dims_send_image, which a failure with no error image never reaches. The pool cleanup records every request whatever path it takes.
Port 8000 serves images. Port 8001 serves /metrics and /dims-status/, so a Service exposes 8000 and a scrape targets 8001. DIMS_METRICS_ENABLED defaults to off, so the image publishes no metrics until an operator turns it on. Behind an ingress controller httpd reads the controller address rather than the caller address, so Require ip cannot separate a scrape from a caller. A second listener moves the decision to the network. The Server image job scrapes 8001 and checks for a module counter and an httpd counter.
A page for /metrics and a page for DimsMetricsEnabled, plus rows in the endpoints table, the defaults table, and the sidebar. dims_build_info and dims_start_time_seconds report the build and the restart time. DimsStatusVerbose off drops the build gauge, the same way it drops the version lines from /dims-status/. A label value escapes a backslash, a double quote, and a newline. The library versions come from ImageMagick and libcurl. dims3.md and dims4.md state that the client segment is a legacy of those two endpoints and that /dims5/ dropped it.
dims_source_fetch_total and dims_source_fetch_errors_total by libcurl code. dims_netguard_refusals_total by reason, and dims_allowlist_checks_total by mode and result. dims_signature_checks_total by endpoint and result, and dims_eurl_decrypt_total. dims_operations_total, dims_operation_failures_total, and dims_operation_duration_seconds per operation. dims_imagemagick_exceptions_total by kind and severity. dims_overlay_cache_lookups_total, dims_overlay_cache_evictions_total, dims_error_images_total, and dims_error_image_failures_total. An operation counts one call. A multi-frame source runs the command loop once per frame, and strip and format run without a command, so the trigger label separates the two. apr_time_now costs 15.7 ns on this host, measured over 20 million calls. Two per operation call adds about 31 ns against a transform measured in milliseconds. The allowlist reports at two sites. handler.c refuses a /dims3/ request before the fetch, and the guard runs its own check at fetch time under DimsAllowlistSigned.
Fourteen families and their labels join the endpoint page.
test/endurance/observe holds a Prometheus and a Grafana that join the soak network and scrape the server on port 8001. up.sh starts them and prints the dashboard address. Grafana listens on every interface, so another machine reaches it. The dashboard has six sections: the overview, the request path, the source fetch, the image processing, the guard and the signature, and the resources. run.sh sets DIMS_METRICS_ENABLED=on, so a run publishes its metrics.
The module answers a conditional request with 304 through ap_meets_conditions. The response code list held no slot for it, so a caching client counted under "other". The dashboard splits its error ratio in two. A 4xx is the request the caller sent, and a 5xx is the module or its source failing. One number covering both reads as a fault when a caller sends a bad signature.
ImageMagick loads its coders with dlopen and unloads them before the process ends. Valgrind discards the symbols for unloaded code, so every frame inside ImageMagick printed as ???. --keep-debuginfo=yes holds them. A leak now names the function and the line: AcquireSemaphoreInfo through NewMagickWand, and GetMagicInfo through MagickReadImageBlob.
beetlebugorg
force-pushed
the
feat/metrics-endpoint
branch
from
September 2, 2026 00:43
0db6935 to
e6289d8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A
dims-metricshandler serves Prometheus text 0.0.4 at a location that sets it.DimsMetricsEnabledis the switch and defaults toOff.src/metrics.cholds a shared memory block that every worker adds to with 64-bit atomics, sized once in post config fromAP_MPMQ_HARD_LIMIT_DAEMONS.src/metrics_report.cwalks it and writes the exposition.dims_handlerregisters a pool cleanup per request, so every return path records once.The endpoint reports 27 families:
dims_requests_total{endpoint,outcome},dims_responses_total{endpoint,code},dims_requests_in_flight{endpoint},dims_request_duration_seconds{endpoint}dims_source_fetch_duration_seconds,dims_source_bytes,dims_source_bytes_total,dims_source_format_total{format},dims_origin_responses_total{code}dims_imagemagick_duration_seconds,dims_source_frames,dims_output_bytes,dims_output_bytes_total,dims_output_format_total{format}dims_imagemagick_resource_bytes{resource},dims_imagemagick_resource_max_bytes{resource},dims_imagemagick_resource_limit_bytes{resource},dims_process_resident_bytes,dims_process_resident_max_bytes,dims_process_virtual_bytes,dims_workersdims_httpd_workers{state},dims_httpd_processes,dims_httpd_processes_limit,dims_httpd_threads_per_process,dims_httpd_connections{state},dims_httpd_generationdims_start_time_seconds,dims_build_infoCounters and histograms come from the block. The ImageMagick resources and the process memory are per worker: each child writes its own slot when a request ends, and the endpoint sums the live slots and reports the widest one. The httpd families read the scoreboard at scrape time.
dims_stats_recgoes away, anddims_status_handlersums the requests table for the four numbers it prints.The container serves images on 8000 and metrics on 8001, with
DIMS_METRICS_ENABLEDoff.Why
/dims-status/prints four counters in prose. A Prometheus scrape of that page returns no series. The request record already holds the duration, the source size, the origin status, the frame count, and the output size, so the endpoint reports them without new measurement.OTLP is push only. Its specification does not define a mode where a server exposes metrics for a collector to fetch. Scraping is the supported path into OpenTelemetry, and the compatibility specification states that the collector adds the resource attributes from the scrape target.
Folding
dims_stats_recinto the block fixes two defects.dims_initsetsuccess_countto 1 at startup, so the endpoint reported one success before it served a request. The four increments sat indims_send_image. A failure on a server with no error image returns before that function, so those requests went uncounted.Port 8001 exists because
Require ipreads the ingress controller address rather than the caller address. A second listener moves the decision to the network.Verify
305 cases pass, every golden file reproduces, and 114 sanitizer cases report no finding.
test/http/test_metrics.ccovers the format, each metric group, and the directive.test/unit/test_metrics.ccovers the bucket search and the index maps.test/http/test_status.casserts the two endpoints agree.Against the built image:
I measured 404 with the image defaults and 200 with the variable on. Port 8000 returns 404 for
/metricsin both cases.Breaking
/dims-status/reports zero successes on a server before its first request, where it reported one. It also counts a failed request that returns no image, which it skipped before.