…VIDIA#534) * feat(observability): implement M7 TELEM05-08 telemetry validations Add storage capacity/performance and NVLink GPU/switch telemetry checks for issues NVIDIA#363-NVIDIA#366 (TELEM-XX-05 through TELEM-XX-08). - Wire TELEM05-08 in observability suite with validation classes - Add AWS probes: EBS performance via CloudWatch; provider-hidden for capacity and NVLink planes not exposed to EC2 tenants - Add my-isv demo scripts and provider config steps - Extend unit tests and mark test-plan entries approved Closes NVIDIA#363 Closes NVIDIA#364 Closes NVIDIA#365 Closes NVIDIA#366 Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com> * fix(observability): raise OBS05 telemetry delivery budget to 600s The telemetry delivery latency check (OBS05-01) enforced a 300s budget, but CloudWatch EC2 metrics land on a 1-minute cadence plus ingestion delay, so the freshest queryable datapoint is routinely 5-6 minutes old. This produced false failures on real AWS runs (observed 335s > 300s). Raise the realistic budget to 600s at both enforcement points (suite validation config and the AWS script arg/default), update the AWS probe unit tests, and document why the 120s NSRG target is not achievable via CloudWatch on EC2. The 600s bound still flags a genuinely stalled telemetry pipeline. Signed-off-by: Alexandre Begnoche <abegnoche@nvidia.com> * refactor(observability): single-source OBS05 delivery budget and fix fail-fast scan Declare the 600s OBS05 delivery budget once in the suite settings and reference it via {{max_delivery_seconds}} from both the validation check and the AWS step arg, instead of hand-syncing three literal copies. The AWS-specific CloudWatch rationale now lives only in the AWS script; the suite comment keeps the provider-neutral facts. Also widen the CloudWatch scan window past the threshold: with both at 600s, a stale-but-present datapoint older than the budget was invisible to the scan, so a stalled pipeline burned the full poll budget and failed with a vague "no datapoints" error instead of failing fast with the precise "exceeds threshold" message. Add a unit test covering the fail-fast path and decouple existing tests from the production budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Alexandre Begnoche <abegnoche@nvidia.com> * fix(observability): address CodeRabbit TELEM review findings Recompute EBS performance discovery after CloudWatch poll refresh, and parameterize storage/NVLink pass-message labels so reports match the validated plane. Signed-off-by: Alexandre Begnoche <abegnoche@nvidia.com> --------- Signed-off-by: Alexandre Begnoche <abegnoche@nvidia.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Uh oh!

TELEM-XX-05: Verify storage resource capacity metrics are available (used/free/total) #363

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions