fix(telemetry): make metrics best-effort so telemetry never crashes storage ops - #125
Draft
jayya2 wants to merge 1 commit into
Draft
fix(telemetry): make metrics best-effort so telemetry never crashes storage ops#125jayya2 wants to merge 1 commit into
jayya2 wants to merge 1 commit into
Conversation
…torage ops Metric/trace exporter failures, unreachable telemetry processes, and OpenTelemetry incompatibilities could raise from the telemetry path and crash daemon upload/download jobs. Make telemetry best-effort: - Disable metrics/traces on any provider/exporter construction failure and cache the disabled state so (possibly expensive) construction isn't retried. - Latch the synchronous record path so a failing record disables metrics for the storage client instead of crashing the storage operation. - Skip metrics initialization entirely when `MSC_TELEMETRY_DISABLED` or `OTEL_SDK_DISABLED` is set. - Fail fast with a clear error when the installed OTLP exporter is too old for the `_otlp_mtls_vault` mTLS client-certificate options instead of an opaque TypeError. Adds unit tests for the disable-on-failure paths, the record latch, and the env kill-switches; documents disabling telemetry. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Metric/trace exporter failures, unreachable telemetry processes, and OpenTelemetry
incompatibilities could raise from the telemetry path and crash daemon upload/download jobs.
This makes telemetry best-effort:
state so (possibly expensive) construction isn't retried.
instead of crashing the storage operation.
MSC_TELEMETRY_DISABLEDorOTEL_SDK_DISABLEDis set._otlp_mtls_vaultmTLS client-certificate options, instead of an opaque
TypeError.Adds unit tests (disable-on-failure paths, record latch, env kill-switches) and documents disabling telemetry.
This is one half of a split (see the companion PR that declares explicit
exposedmethods on thecross-process metric instrument proxies); the two are independent.
Checklist
.release_notes/.unreleased.md🤖 Generated with Claude Code