feat: expose Prometheus /metrics endpoint for usage dashboards#102
Draft
dobby-coder[bot] wants to merge 2 commits intomainfrom
Draft
feat: expose Prometheus /metrics endpoint for usage dashboards#102dobby-coder[bot] wants to merge 2 commits intomainfrom
dobby-coder[bot] wants to merge 2 commits intomainfrom
Conversation
Adds a Prometheus text-format `GET /metrics` endpoint covering the metrics requested in #101: * cryptify_uploads_total{channel} * cryptify_upload_bytes_total{channel} * cryptify_storage_bytes (gauge, sampled periodically from data_dir) * cryptify_active_files (gauge, same source) * cryptify_expired_files_total (counter, purged-before-finalized) The channel label is derived from request headers: 1. X-Cryptify-Source explicit header 2. Authorization: Bearer / X-Api-Key -> "api" 3. Origin -> "staging-website" / "website" 4. User-Agent substring -> "outlook" / "thunderbird" 5. fallback "unknown" Values are sanitized (lower-case [a-z0-9_-], max 32 chars) to prevent label-injection and cardinality blowup. Storage gauges are sampled by a background task that walks data_dir every `metrics_scan_interval_secs` (default 60, configurable). Dashboard JSON ready for import into the Scaleway Grafana instance is shipped under `docs/grafana/`, alongside a Prometheus scrape-config example. No authentication on /metrics; restrict via firewall / proxy allow-list (documented in README and docs/grafana/README.md). Refs #101
Contributor
|
/dobby fix conflicts |
Contributor
Author
|
Dobby has seen the request! Dobby will fix the conflicts right away, sir! Dobby is very good at resolving merge conflicts — Dobby will have this PR clean and ready in no time! 🧹 |
Integrate main-branch additions (chunk_size config, per-upload/rolling rate limits, API key detection, /usage endpoint) alongside the metrics feature from this branch. Both feature sets are preserved and work together. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Author
|
Conflicts resolved and pushed. Here's what happened: Conflicting files: Root cause: Main merged chunk-size limits, per-upload/rolling rate limiting, API key detection, and a Resolution: Kept both feature sets — all metrics fields/logic alongside the new rate-limiting, chunk-size, and API key features. Specifically:
Verification:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the server-side half of #101: Cryptify now exposes a Prometheus text-format
GET /metricsendpoint that Grafana on Scaleway can scrape to render the usage dashboards the issue asks for.Metrics:
cryptify_uploads_total{channel}cryptify_upload_bytes_total{channel}cryptify_storage_bytesdata_dircryptify_active_filesdata_dircryptify_expired_files_totalChannel detection
channelis derived from request headers in this priority order:X-Cryptify-Source(explicit header, sanitized)Authorization: Bearer …/X-Api-Key→apiOrigin→staging-website/websiteUser-Agentsubstring →outlook/thunderbirdunknownAll label values are lower-cased, restricted to
[a-z0-9_-], and capped at 32 chars to prevent label-injection or cardinality explosions.Storage gauges
A background task walks
data_direverymetrics_scan_interval_secs(new config option, default60) and updates the two gauges. This avoids touching the upload hot path.Dashboard
docs/grafana/postguard-usage.jsonis a ready-to-import dashboard covering:environmentlabel)docs/grafana/README.mdcontains a reference Prometheus scrape config.Why draft
/metricsis unauthenticated. The README and Grafana docs call out that access must be restricted at the firewall / reverse proxy. Confirm that matches the Scaleway / Procolix network policy before merging.X-Cryptify-Source. Until follow-up PRs land in those repos they fall back to theUser-Agentrule, which is approximate. Filing follow-ups as separate issues.cryptifybinary requires a reachable PKG server at startup (documented gotcha). Unit tests cover the metrics module comprehensively (13 tests, all green) but an integration run on a real staging deploy is worth doing before promoting out of draft.Test plan
cargo check— cleancargo test— 13 new tests inmetrics::tests, all passcargo clippy --all-targets— no new warnings (one pre-existing warning insrc/email.rs:225, left alone)/metricswith Prometheus, importdocs/grafana/postguard-usage.json, confirm counters increment when uploading via postguard.eu / staging / addons/metricsto the Prometheus segmentFollow-ups (not in this PR)
X-Cryptify-Sourceheader (outlook / thunderbird)cryptify_upload_bytes_totalshould be emitted fromupload_chunkinstead offinalize(trade-off: finer-grained vs. includes abandoned uploads)Refs #101
Co-Authored-By: Claude Opus 4.7 noreply@anthropic.com