feat(observability): attribute dashboard panels per cluster and server - #151
Open
gabriele-wolfox wants to merge 6 commits into
Open
gabriele-wolfox wants to merge 6 commits into
gabriele-wolfox wants to merge 6 commits into
Conversation
gabriele-wolfox
force-pushed
the
dev/49
branch
4 times, most recently
from
August 27, 2026 08:14
deb2469 to
c01ba17
Compare
gabriele-wolfox
force-pushed
the
dev/49
branch
3 times, most recently
from
September 21, 2026 12:05
0b3dc9b to
a6e1bc8
Compare
The Grafana dashboard was only validated against a single server and cluster, where sums and maxima are unambiguous. With several servers and clusters it folded independent entities into single misleading values and could not tell them apart. Add a cluster_name attribute to the plugin backup metrics (klio.plugin.backup.*) so backup panels can be attributed per cluster even when several clusters share a namespace. Rework the dashboard so every aggregation groups by its identifying label and nothing is folded silently: - Identify servers by the OpenTelemetry service.name instead of the pod host name, which collides when two servers share a name in different namespaces. - Scope server panels by service.name and cluster_name rather than the namespace, so a cluster backed up cross-namespace is attributed to the right server. - Group every panel by cluster, server and tier, and identify each series in its legend, so per-tier backup and WAL state is never folded (tier-1 and tier-2 have independent relay progress and retention). - Render the PostgreSQL timelines as stepped time series rather than a single current value, so a promotion or failover is visible as the step where the line jumps. - Derive a cluster label from the Kopia snapshot source so the base snapshot panels are per cluster too. Update the metric catalog descriptions and the OpenTelemetry and Grafana documentation, and regenerate the committed dashboard JSON. Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
Regenerate the three dashboard section screenshots against a multi-server, multi-cluster, two-tier environment so they reflect the reworked panels: per-cluster and per-tier series, servers identified by service name, and the PostgreSQL timelines rendered as stepped time series. Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
Backup failures did not show on the dashboard: the runs counter series for an outcome and failure category only appears on the first backup of that kind, so it is born at 1 and rate()/increase() (and the success-ratio panel built on them) cannot see that first event. Seed the runs counter at 0 for every outcome and failure category, and the in-progress gauge, when the plugin starts, so a first failure is a visible 0->1, per the Prometheus "avoid missing metrics" guidance: https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
Spread the horizontal stat orientation across the server tiles so values stay readable with multiple clusters selected. Replace the base-snapshot and retained-backup bar gauges with compact stat panels, and render backup and WAL LSNs as tables that split each position into high and low 32-bit hex halves (PostgreSQL X/Y), sizing the columns to fit. Print exact snapshot file counts, and count backup runs per bucket with changes() so the outcome bars render at any range. Regenerate the committed dashboard JSON. Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
Regenerate the three dashboard section screenshots so they reflect the reworked panels, with tier1 and tier2 series populated across the server and WAL replication sections. Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
gabriele-wolfox
force-pushed
the
dev/49
branch
from
September 21, 2026 13:12
a6e1bc8 to
b6adb90
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Grafana dashboard was only validated against a single server and cluster, where sums and maxima are unambiguous. With several servers and clusters it folded independent entities into single misleading values and could not tell them apart.
Add a
cluster_nameattribute to the plugin backup metrics (klio.plugin.backup.*) so backup panels can be attributed per cluster even when several clusters share a namespace.Rework the dashboard so every aggregation groups by its identifying label and nothing is folded silently:
service.nameandcluster_namerather than the namespace, so a cluster backed up cross-namespace is attributed to the right server.Update the metric catalog descriptions and the OpenTelemetry and Grafana documentation, and regenerate the committed dashboard JSON.
Refine the reworked panels for multi-cluster readability:
changes()so the per-outcome bars render at any dashboard range.Refresh the three dashboard section screenshots, with tier-1 and tier-2 series populated across the server and WAL replication sections.
Assisted-by: Claude
Closes #149