Add SLO checks with a SQLAlchemy read/write workload#116
Open
vgvoleg wants to merge 6 commits into
Open
Conversation
Introduce a parallel read/write SLO workload built on the ydb_sqlalchemy dialect (SQLAlchemy Core and ORM modes) and wire it into ydb-slo-action via a label-gated GitHub workflow. - tests/slo: workload runner, Dockerfile, entrypoint, requirements, README - .github/workflows/slo.yml: build current+baseline images, run init@v2 and publish report@v2 on PRs labelled "SLO"
The dialect integration tests now live in tests/integration/ alongside tests/slo/, so the repo no longer has both a test/ and a tests/ directory. tox.ini (lint + dialect pytest paths) and setup.cfg (profile_file) are updated accordingly.
🌋 SLO Test Results🟢 2 workload(s) tested — All thresholds passed
Generated by ydb-slo-action |
The dialect runs in AUTOCOMMIT, so each single-statement read/write already goes through the YDB SDK's retry_operation_sync inside ydb-dbapi. The workload now performs one attempt per operation and records any surfaced exception as a real SLO failure, instead of a broad app-level retry loop that masked non-retryable errors. Removes the now-unused timeout/max-retries flags.
Align workload_duration and read/write RPS with ydb-python-sdk's tests/slo workflow. extra-nodes stays disabled to fit a GitHub-hosted runner.
…cluster, 600s, 1000/100 rps) Run the workload job on the large-runner-sqlalchemy self-hosted runner with the full YDB cluster (all compose profiles), and align workload_duration and read/write RPS with ydb-python-sdk's tests/slo workflow. The report job stays on ubuntu-latest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds SLO (Service Level Objective) testing on top of
ydb-platform/ydb-slo-action, following the ydb-python-sdk SLO example but expressed entirely in terms of SQLAlchemy.Workload (
tests/slo/)A parallel read/write load generator driving the
ydb_sqlalchemydialect:SELECT ... WHERE object_id = :idfor a random id;UPSERT INTO ... VALUES (...)for a fresh id;Two modes, selected by
WORKLOAD_NAME/--mode:coreConnection.execute(select())Connection.execute(upsert())ormSession.get(KeyValueRow, id)Session.execute(upsert())+ commitMetrics are emitted via OTLP with names matching the action's default
metrics.yaml(sdk_operations_total,sdk_operation_latency_p{50,95,99}_seconds,sdk_retry_attempts_total, ...).Workflow (
.github/workflows/slo.yml)Runs on PRs labelled
SLO:current(PR) andbaseline(merge-base) workload images;ydb-slo-action/init@v2for thecoreandormworkloads in parallel;ydb-slo-action/report@v2and gates the PR on regressions.The cluster is trimmed to fit a GitHub-hosted runner via
disable_compose_profiles: extra-nodes(chaos and telemetry stay enabled).How to run
Label this PR with
SLOto trigger the checks. Locally:Notes
reportjob needspull-requests: write, which same-repo PRs have. For fork PRs the report can be moved to a separateworkflow_run-triggered workflow.tests/slo/is outside the existingtest/lint scope, so it doesn't affect the style/tests workflows.