Skip to content

fix(telemetry): make metrics best-effort so telemetry never crashes storage ops - #125

Draft
jayya2 wants to merge 1 commit into
mainfrom
claude/telemetry-best-effort
Draft

fix(telemetry): make metrics best-effort so telemetry never crashes storage ops#125
jayya2 wants to merge 1 commit into
mainfrom
claude/telemetry-best-effort

Conversation

@jayya2

@jayya2 jayya2 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Description

Metric/trace exporter failures, unreachable telemetry processes, and OpenTelemetry
incompatibilities could raise from the telemetry path and crash daemon upload/download jobs.
This makes telemetry best-effort:

  • Disable metrics/traces on any provider/exporter construction failure and cache the disabled
    state so (possibly expensive) construction isn't retried.
  • Latch the synchronous record path so a failing record disables metrics for the storage client
    instead of crashing the storage operation.
  • Skip metrics initialization entirely when MSC_TELEMETRY_DISABLED or OTEL_SDK_DISABLED is set.
  • Fail fast with a clear error when the installed OTLP exporter is too old for the _otlp_mtls_vault
    mTLS client-certificate options, instead of an opaque TypeError.

Adds unit tests (disable-on-failure paths, record latch, env kill-switches) and documents disabling telemetry.

This is one half of a split (see the companion PR that declares explicit exposed methods on the
cross-process metric instrument proxies); the two are independent.

Checklist

  • Development PR
    • .release_notes/.unreleased.md
      • Notable changes to the client from this PR have been added.

🤖 Generated with Claude Code

…torage ops

Metric/trace exporter failures, unreachable telemetry processes, and OpenTelemetry
incompatibilities could raise from the telemetry path and crash daemon upload/download
jobs. Make telemetry best-effort:

- Disable metrics/traces on any provider/exporter construction failure and cache the
  disabled state so (possibly expensive) construction isn't retried.
- Latch the synchronous record path so a failing record disables metrics for the
  storage client instead of crashing the storage operation.
- Skip metrics initialization entirely when `MSC_TELEMETRY_DISABLED` or
  `OTEL_SDK_DISABLED` is set.
- Fail fast with a clear error when the installed OTLP exporter is too old for the
  `_otlp_mtls_vault` mTLS client-certificate options instead of an opaque TypeError.

Adds unit tests for the disable-on-failure paths, the record latch, and the env
kill-switches; documents disabling telemetry.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 673d879b-92a2-4c5c-b368-c4132c88a851

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/telemetry-best-effort

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant