otel-demo: upgrade docker-compose integration to 3.0.0 - #21
Closed
kylehounslow wants to merge 6 commits into
Closed
Conversation
Bump IMAGE_VERSION and DEMO_VERSION to 3.0.0. Replace Locust load-generator env with k6 (LOAD_GENERATOR_VUS, K6_TARGET_URL). Remove env for services not in the 3.0 core demo (product-reviews, kafka, llm). Rename postgresql service to astronomy-db with astronomy_user/monitoring_user scheme; add AD_PROMETHEUS_PORT, OTEL_SERVICE_NAMESPACE, TELEMETRY_DOCS_*, FRONTEND_PROXY_ADDR. Bump flagd/postgres/valkey image pins to the 3.0 set. Rewrite docker-compose.otel-demo.yml to the 3.0.0 core services: drop accounting, fraud-detection, kafka, llm, product-reviews; add telemetry-docs; adopt upstream 3.0 healthchecks, k6 load-generator, and per-service service.criticality resource attributes. Preserve stack customizations: network/logging anchors, container names, memory limits, restart policy, telemetry pointed at the stack collector, image refs, unbound port publishing, and the local-backends alerting-init service. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
Update demo.flagd.json and postgresql/init.sql to 3.0.0. init.sql now provisions astronomy_db with astronomy_user and monitoring_user; flags for removed services drop out. Add product-catalog otel-config.yml mounted by the product-catalog service in 3.0. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
Remove the product-reviews panels from the astronomy-shop dashboard and drop fraud-detection and product-reviews from the service-telemetry dashboard variable list. Repoint the product catalog throughput panel from the removed npgsql (.NET) counters to the Go client metric db_client_operation_duration_seconds_count. Drop product-reviews and accounting from the otel-demo alert rule service matchers. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
Keep this PR scoped to docker-compose. The charts/observability-stack/files copies of the astronomy dashboards and otel-demo alert rules feed the Helm init-dashboards and cortex-rules configmaps, but the opentelemetry-demo subchart is still pinned at 0.40.8. Shipping 3.0-service-set dashboards and alerts into the chart ahead of the subchart bump leaves the Helm path half-updated (astronomy-db and dropped-service references against a pre-3.0 deployment). These belong with the helm 3.0 follow-up that also bumps the subchart. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
The 3.0.0 frontend-proxy image's envoy bootstrap defines STRICT_DNS clusters for CHATBOT_HOST/PORT, FIREPIT_HOST/PORT and OPAMP_SERVER_HOST/UI_PORT. This stack does not run those services; without the vars the socket addresses are empty, proto validation fails, and the proxy crash-loops, which also blocks load-generator (depends on frontend-proxy healthy). Add the vars to .env with the upstream 3.0.0 defaults and pass them into the frontend-proxy container. The hosts do not resolve here, so the clusters stay unhealthy at runtime without blocking startup, matching the existing grafana/jaeger handling. Signed-off-by: Kyle Hounslow <kylhouns@amazon.com>
|
✅ Docs preview build completed successfully! All tests passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #20 (closed on force-push). Rebased onto current
mainso the diff stays clean; the only upstream change since the branch point was opensearch-project#432 (quickstart docs), which touches none of the eight files here.What
Upgrades the docker-compose otel-demo integration from 2.2.0 to 3.0.0. Required to stay on the otel-demo adopter list (open-telemetry/opentelemetry-demo#3720). This is a breaking upgrade, not a version bump: 3.0.0 renamed
docker-compose.ymltocompose.yamland split the core demo from afullprofile.Anchored to the upstream
open-telemetry/opentelemetry-demo3.0.0tag, diffed against2.2.0. Draft, and scoped to docker-compose only. The diff touches only.env,docker-compose.otel-demo.yml, and files underdocker-compose/. Nothing undercharts/(Helm), k8s, or AWS CLI/CDK. Those are follow-ups.Service changes (24 to 20)
accounting,fraud-detection,kafka,llm,product-reviews. In 3.0 the first three moved tocompose.full.yaml;llmandproduct-reviewsare gone.postgresqltoastronomy-dbwith the 3.0astronomy_user/monitoring_userscheme. product-catalog now readscatalog.productsviapostgres://astronomy_user:...@astronomy-db/astronomy_db.telemetry-docs.LOAD_GENERATOR_VUS,K6_TARGET_URLreplace theLOCUST_*vars)..env(GRAFANA_IMAGE,JAEGERTRACING_IMAGE,OPENSEARCH_IMAGE,TRACETEST_IMAGE*,COLLECTOR_CONTRIB_IMAGE, java/cpp agent versions) were left untouched. The stack provides its own backends and does not reference them; prune separately if desired.Stack customizations preserved on every ported service:
observability-stack-network, shared logging, container names, memory limits, telemetry pointed at the stack collector (OTEL_EXPORTER_OTLP_ENDPOINT/depends_on: otel-collector), and the stack-specificotel-demo-alerting-rules-monitors-initservice.Files
.env,docker-compose.otel-demo.ymldocker-compose/opentelemetry-demo/src/flagd/demo.flagd.json,.../postgresql/init.sql, new.../product-catalog/otel-config.ymldocker-compose/opensearch-dashboards/init/dashboard-astronomy-shop.ndjson,.../dashboard-astronomy-service-telemetry.ndjson,docker-compose/prometheus/rules-otel-demo/otel-demo-alerts.ymlThe astronomy dashboards and otel-demo alert rules are mirrored into
charts/observability-stack/files/, which feed the Helm init-dashboards and cortex-rules configmaps. Those copies are deliberately left unchanged here: theopentelemetry-demosubchart is still pinned at 0.40.8, so shipping 3.0-service-set dashboards and alerts into the chart ahead of the subchart bump would leave the Helm path half-updated (astronomy-db and dropped-service references against a pre-3.0 deployment). The chart-side dashboard and alert updates land with the helm 3.0 follow-up that also bumps the subchart.Validation
Static only, confirmed this pass:
INCLUDE_COMPOSE_OTEL_DEMO=docker-compose.otel-demo.yml docker compose --env-file .env config -qexits 0, no warnings.ghcr.io/open-telemetry/demo:3.0.0-*;DB_CONNECTION_STRINGandK6_TARGET_URLinterpolate correctly.Not yet validated (needs a live bring-up; gates promotion off the fork):
docker compose upof the 3.0 stack end to end with telemetry flowing. A live 2.2.0 stack runs on the dev host and was deliberately not disturbed; the e2e run needs an isolated compose project.db_client_operation_duration_seconds_count. The exact metric name and labels emitted by the 3.0 Go service need live confirmation.astronomy-dbinit seeding and product-catalog reads under the new scheme.Follow-ups (out of scope here)
opentelemetry-demosubchart to a 3.0-compatible version, apply the astronomy-db rename, and update thecharts/observability-stack/files/dashboard and alert copies to match this PR's docker-compose versions.AD_PROMETHEUS_PORT, astronomy-dbmonitoring_user); that lives in the main compose/collector config, not this file.