feat(webapp,run-engine,core,clickhouse): surface total concurrency in metrics and dashboard - #4830
Conversation
… metrics and dashboard Queues with a totalConcurrencyLimit now report how they use it. The gauge pipeline emits total running and the stored cap, ClickHouse aggregates them into the queue metrics tiers, the queues list gets a Total column, the queue detail page charts total running against the cap, and the per-key table shows each key's effective limit including per-key overrides. Queue retrieve and list API responses include the same totals.
🦋 Changeset detectedLatest commit: 8f5ad5d The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…tore views on rollback The CK enqueue gauges (fast path and queued path) now sample total running and the stored cap, so metric buckets fed only by enqueues no longer record zero totals. The migration's down section recreates the pre-existing materialized view definitions so ingestion keeps flowing after a rollback. The per-key table reads only the page's overrides with one HMGET instead of loading the queue's whole override hash.
1 participant
Summary
Queues with a
totalConcurrencyLimitnow show how they use it, in the dashboard and the API. Stacked on #4829.The queues list gains a Total column showing runs in flight across all concurrency keys against the queue's total cap, turning warning-colored when the cap is reached. The queue detail page gains a "Total concurrency" chart plotting total running against the effective cap over time, and the per-key table gains a Limit column that shows each key's effective concurrency limit, highlighting keys with a per-key override. Queue retrieve and list API responses include the same totals under
concurrency.total.Design
The queue metrics gauge (emitted from the dequeue and enqueue Lua scripts) is extended with two fields: total running (the group concurrency set's cardinality) and the raw stored total limit. They flow through the existing metrics stream into the ClickHouse raw table and are aggregated as
max_total_running/max_total_limitin the 10s and 5m rollup tiers, queryable from the dashboard's TRQL widgets. The per-key rollup additionally records each key's effective limit so overrides are visible historically.Reading the live numbers is O(pages shown): the list presenter batches one SCARD per queue that has a cap, and the per-key table reads the queue's override hash once per page.