feat(app): show GPU metrics in log side panel infrastructure section - #2897
Draft
MikeShi42 wants to merge 5 commits into
Draft
feat(app): show GPU metrics in log side panel infrastructure section#2897MikeShi42 wants to merge 5 commits into
MikeShi42 wants to merge 5 commits into
Conversation
Add GPU utilization and GPU memory utilization charts to the Infrastructure tab of the log/span side panel, using OTel hardware semantic conventions (hw.gpu.*). - Add useGpuMetricsAvailability hook for cheap metric existence check (queries MetricName values from gauge table, cached 5 min) - Add GpuInfraSection component with per-GPU series via hw.id groupBy - Add getGpuCorrelationWhere to build resource correlation filter (prefers k8s.node.name, falls back to host.name) - Section is fully hidden when no GPU metrics exist for the correlated resource; partial availability renders only available charts - GPU utilization chart filters to hw.gpu.task:general (or unset) to avoid mixing encoder/decoder series HDX-5102 Co-authored-by: Mike Shi <mike@hyperdx.io>
🦋 Changeset detectedLatest commit: 0e95137 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThe PR adds availability-gated GPU utilization charts to the log/span Infrastructure panel. The filtered metric discovery remains capped and can hide supported charts on sources with a sufficiently broad GPU metric namespace.
Confidence Score: 4/5The PR is not yet safe to merge because capped GPU metric discovery can hide supported charts and, in some cases, the entire GPU section. The availability lookup narrows results to GPU metrics but still accepts the metadata helper's 20-value aggregation limit, so supported metric names can be omitted from the set used to decide whether charts render. Files Needing Attention: packages/app/src/hooks/useGpuMetricsAvailability.ts
|
| Filename | Overview |
|---|---|
| packages/app/src/hooks/useGpuMetricsAvailability.ts | Adds GPU metric discovery, but the metadata aggregation can truncate the filtered metric-name set and hide supported charts. |
| packages/app/src/components/DBInfraPanel.tsx | Adds availability gating and generalized chart configuration; the control-handler assertions remain an incomplete non-blocking type-safety fix. |
| packages/app/src/components/infraCorrelations.ts | Defines the GPU correlation, chart filters, formatting, and per-device grouping. |
| packages/app/src/components/tests/infraCorrelations.test.ts | Covers GPU descriptors, chart specifications, grouping expressions, and primary-metric availability resolution. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
Event[Log or span with k8s.node.name] --> Availability[Query hw.gpu.* metric names]
Availability --> Gate{Supported metric returned?}
Gate -->|Yes| Charts[Render available GPU charts]
Gate -->|No| Hidden[Hide GPU section]
Availability -. 20-name cap .-> Gate
Reviews (5): Last reviewed commit: "fix(app): drop broken memory fallback, f..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Use specific type assertions instead of 'as any' to stay within the max-warnings threshold. Fix import sort order in test file. Co-authored-by: Mike Shi <mike@hyperdx.io>
Contributor
E2E Test Results✅ All tests passed • 282 passed • 1 skipped • 1069s
Tests ran across 4 shards in parallel. |
Use the stable NOW constant from config instead of new Date() in the GPU infra section's date range calculation, matching the project's date hygiene rules. Co-authored-by: Mike Shi <mike@hyperdx.io>
Structural: - Extend InfraChartSpec with optional groupBy, where, metricType, and fallback fields so GPU registers as a descriptor rather than a separate component - Delete GpuInfraSection.tsx; InfraSubpanelGroup now handles both k8s and GPU charts via the descriptor data - Add requiresMetricAvailability flag to InfraCorrelation; gated groups only render when metric existence is confirmed - Add AvailabilityGatedGroup wrapper that checks availability before rendering Bug fixes: - Fix _exists_ syntax (unsupported) → hw.gpu.task:* per query parser - Push MetricName:hw.gpu.* prefix filter into the availability query so limit doesn't produce false negatives on metric-heavy nodes - Check both gauge and sum tables for availability - Drop host.name fallback (unreachable: tab requires k8s attributes) Acceptance criteria: - Series grouped by concat(hw.id, hw.name, hw.model) for richer labels - hw.gpu.memory.usage / hw.gpu.memory.limit fallback via ratio chart when hw.gpu.memory.utilization isn't emitted - resolveChartAvailability tested for primary, fallback, none, and partial cases Minor: - Use live new Date() (eslint-disable) matching sibling InfraSubpanelGroup - Move GPU_UTILIZATION_NUMBER_FORMAT to ChartUtils for consistency - Remove empty select comment (now annotated in the hook) HDX-5102 Co-authored-by: Mike Shi <mike@hyperdx.io>
Remove the hw.gpu.memory.usage / hw.gpu.memory.limit fallback: convertV1ChartConfigToV2 drops seriesReturnType for metrics, the renderer discards the second series, and Sum uses counter-increase semantics on a non-monotonic UpDownCounter. All three failures are in the renderer and out of scope for this PR. Changes: - Remove InfraChartFallback type and fallback field from InfraChartSpec - Simplify resolveChartAvailability to return boolean (available / not) - Remove sum-table query from useGpuMetricsAvailability (halves cost) - Simplify buildChartConfig (no mode parameter) - Fix 40px empty-div gap: return null from the correlation map entry when both metricsGroup and timeline render nothing, so no empty flex child is emitted into Stack Follow-up: HDX-5102 — support ratio charts over Sum metrics for GPU memory fallback (requires changes to convertV1ChartConfigToV2, renderChartConfig metric select handling, and Sum aggFn projection). HDX-5102 Co-authored-by: Mike Shi <mike@hyperdx.io>
Comment on lines
+53
to
+56
|
|
||
| const { data, isLoading } = useGetKeyValues( | ||
| { chartConfig, keys: ['MetricName'], disableRowLimit: true }, | ||
| { enabled: enabled && !!chartConfig }, |
Contributor
There was a problem hiding this comment.
If a correlated node has more than 20 distinct hw.gpu.* gauge metric names, disableRowLimit still leaves the metadata query's default groupUniqArray(20) cap in place. The result can omit hw.gpu.utilization or hw.gpu.memory.utilization, causing an available chart—or the entire GPU section—to remain hidden.
Knowledge Base Used: App Components and Charts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds GPU utilization and GPU memory utilization charts to the Infrastructure tab of the log/span side panel, using the OpenTelemetry hardware semantic conventions (
hw.gpu.*).Design
GPU registers as a descriptor in
infraCorrelations.tsalongside Pod and Node — not as a separate component. The extendedInfraChartSpectype supports:groupBy— per-chart SQL expressions for multi-GPU series labelingwhere— per-chart Lucene filter (ANDed with the correlation WHERE)metricType—'Gauge'(default) or'Sum'The
InfraCorrelationtype gainsrequiresMetricAvailability: boolean. When set, anAvailabilityGatedGroupwrapper checks metric existence before rendering — the entire group is hidden (returns null) when no GPU metrics exist.Correlation & labeling
k8s.node.name(same as the existing Node group).concat(hw.id, hw.name, hw.model)so multi-GPU nodes show identifiable per-device lines.Metric existence check (
useGpuMetricsAvailability)MetricName:hw.gpu.*prefix filter into the ClickHouse query so the result set is bounded regardless of how many other metrics the source has.useGetKeyValues(5 min staleTime).Charts
hw.gpu.utilizationhw.gpu.task:"general" OR NOT hw.gpu.task:*hw.gpu.memory.utilizationGraceful degradation
Out of scope (follow-up)
The
hw.gpu.memory.usage / hw.gpu.memory.limitfallback from the original issue is deferred. It requires ratio chart support over Sum metrics, which doesn't work today due to three independent issues inconvertV1ChartConfigToV2andrenderChartConfig(seriesReturnType dropped for metrics, second series silently discarded, counter-increase semantics applied to UpDownCounters). Filed as a follow-up on HDX-5102.Screenshots or video
N/A — GPU metrics require an OTel-semconv-compliant GPU collector. The preview demo has no GPU data, so the section remains hidden by design.
How to test on Vercel preview
N/A — non-UI change for environments without
hw.gpu.*metrics.References
Linear Issue: HDX-5102