-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Control Plane latency recommendations for reliable clusters #78769
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
47b55a6
a762eec
abbee2c
e2c0330
1169c18
02e5edc
936bc5d
aa6efad
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * scalability_and_performance/control-plane-latency-recommendations-for-reliable-clusters.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="control-plane-latency_{context}"] | ||
| = Recommended control plane latency for cluster reliability | ||
|
|
||
| Latency between each of the control plane nodes must be less than 15ms to ensure a well performing and reliable cluster. Some of the metrics to keep track of include etcd gRPC requests latency, fysnc latency and any critical alerts. | ||
|
|
||
| The following PromQL queries can help monitor latency requirements: | ||
| [source,terminal] | ||
|
shahsahil264 marked this conversation as resolved.
|
||
| ---- | ||
| histogram_quantile(0.99, sum(rate(grpc_server_handling_seconds_bucket{job=~".*etcd.*", grpc_method!="Defragment", grpc_type="unary"}[5m])) without(grpc_type)) < 0.15 | ||
|
|
||
| avg_over_time(histogram_quantile(0.99, rate(etcd_disk_wal_fsync_duration_seconds_bucket[2m]))[10m:]) < 1 | ||
|
|
||
| ALERTS{severity="critical", alertstate="firing"} > 0 | ||
|
|
||
| ---- | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| [id="control-plane-latency-recommendations-for-reliable-clusters"] | ||
| = Control plane latency recommendations for reliable clusters | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] AsciiDocDITA.ShortDescription: Assign [role="_abstract"] to a paragraph to use it as in DITA. |
||
| include::_attributes/common-attributes.adoc[] | ||
| :context: control-plane-latency-recommendations-for-reliable-clusters | ||
|
|
||
| toc::[] | ||
|
|
||
| This topic provides recommended Control Plane latency for reliable clusters. | ||
|
|
||
| include::modules/control-plane-latency.adoc[leveloffset=+1] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| [id="recommended-control-plane-latency-recommendations-for-reliable-clusters"] | ||
| = Recommended control plane latency recommendations for reliable clusters | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] AsciiDocDITA.ShortDescription: Assign [role="_abstract"] to a paragraph to use it as in DITA. |
||
| include::_attributes/common-attributes.adoc[] | ||
| :context: recommended-control-plane-latency-recommendations-for-reliable-clusters | ||
|
|
||
| toc::[] | ||
|
|
||
| This topic provides the recommended control plane latency to ensure cluster reliability in {product-title}. | ||
|
|
||
| include::modules/control-plane-latency.adoc[leveloffset=+1] | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤖 [error] AsciiDocDITA.ShortDescription: Assign [role="_abstract"] to a paragraph to use it as in DITA.