feat: hardcode ETCD_METRICS=extensive in etcd StatefulSet template - #9192
feat: hardcode ETCD_METRICS=extensive in etcd StatefulSet template#9192shaheen0b111 wants to merge 1 commit into
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: shaheen0b111 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (6)
📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe etcd container in the hosted control plane StatefulSet now sets Suggested reviewers: 🚥 Pre-merge checks | ✅ 11✅ Passed checks (11 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9192 +/- ##
==========================================
+ Coverage 44.56% 44.77% +0.21%
==========================================
Files 774 775 +1
Lines 97003 97210 +207
==========================================
+ Hits 43228 43528 +300
+ Misses 50783 50681 -102
- Partials 2992 3001 +9 see 28 files with indirect coverage changes
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Enable grpc_server_handling_seconds histogram metrics on all HCP etcd pods by setting ETCD_METRICS=extensive unconditionally. This exposes gRPC latency histograms needed for P99/P95 SLO alerting. Ref: ARO-26896
1112c53 to
2b64afe
Compare
|
@shaheen0b111: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Summary
Enable
grpc_server_handling_secondshistogram metrics on all HCP etcd pods by settingETCD_METRICS=extensiveunconditionally in the etcd StatefulSet template.This exposes gRPC latency histograms needed for P99/P95 SLO alerting without requiring any annotation plumbing or controller logic.
Jira: ARO-26896
What changed
Single file:
control-plane-operator/controllers/hostedcontrolplane/v2/assets/etcd/statefulset.yamlAdded
ETCD_METRICS=extensiveenv var to the etcd container, after the existingETCD_ENABLE_PPROFentry.Why
basicmetrics mode which only emits counters (grpc_server_started_total,grpc_server_handled_total)extensivemode additionally emitsgrpc_server_handling_secondshistograms (bucket/count/sum), which are required for computing P99/P95 gRPC request latency SLOsextensivemode unconditionally — the additional histogram metrics have minimal overheadTesting
Tested on ARO-HCP personal dev environment:
ETCD_METRICS=extensiveis set on all 3 etcd pods:Summary by CodeRabbit
Summary by CodeRabbit