Problem Statement
A suggestion rather than a defect report, and entirely yours to weigh — you know the cost of your
CI minutes and we do not.
CI never starts a SurrealDB instance, so every test that needs one skips. That leaves the only
production backend untested in the pipeline, and it is the reason #143 was necessary: the test
that would have caught those three bugs was added in the same PR that introduced them (#139,
tests/integration/test_surrealdb_pinning.py) and has been skipping ever since.
Measured on v3.0.3:
$ grep -rln "SURREALDB_URL" tests/ | wc -l
26
$ grep -ci "surreal" .github/workflows/ci.yml
1 # and that one occurrence is `--cov=surreal_memory`
$ grep -rn "services:" .github/workflows/*.yml
# no output — no workflow starts any service
26 test files are gated on SURREALDB_URL. The test job installs .[dev,server] — not even
the surrealdb extra — and runs pytest tests/ -m "not stress", so all of them skip green.
The parity ratchet added around #139 catches "the method is missing". It cannot catch
"the method is present and returns something that looks right" — which is precisely what
#143 fixed: SELECT VALUE over an array field returned a list of lists, the unwrap heuristic
collapsed it, and the caller iterated a string character by character. No signature was wrong.
Only a live engine shows that.
Proposed Solution
One integration job with a SurrealDB service container. Everything else stays as it is.
Untested. We cannot run your CI, and we have not run this YAML anywhere — it is written
from the SurrealDB image's documented interface and the shape of your existing test job, not
from a green run. Treat it as a starting point, not a verified patch. The image tag, the
health-check invocation and the exact --user/--pass flags are the parts most likely to need
adjusting.
integration:
name: Integration (SurrealDB)
runs-on: ubuntu-latest
timeout-minutes: 15
services:
surrealdb:
image: surrealdb/surrealdb:v3.2.0
ports:
- 8000:8000
options: >-
--user root
--health-cmd "/surreal is-ready --endpoint http://localhost:8000"
--health-interval 5s
--health-timeout 5s
--health-retries 12
env:
SURREAL_USER: root
SURREAL_PASS: root
SURREAL_BIND: 0.0.0.0:8000
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install -e ".[dev,server,surrealdb]"
- name: Run the integration suite
env:
SURREALDB_URL: http://127.0.0.1:8000
SURREALDB_USER: root
SURREALDB_PASS: root
SURREALDB_NS: ci
SURREALDB_DB: ci
run: pytest tests/integration -v --timeout=120 -m "not stress"
If the official image's entrypoint needs an explicit start command in the Actions service
context, the alternative is to run the same container as an ordinary step
(docker run -d -p 8000:8000 …) and poll /health before the test step — less elegant, but it
sidesteps every options: question above.
Use Case
# tests/integration/test_surrealdb_pinning.py:46-51 — today
pytestmark = [
pytest.mark.integration,
pytest.mark.skipif(
not SURREALDB_URL, reason="requires SURREALDB_URL (live SurrealDB >= 3.2.0)"
),
]
With the job above, that marker stops being a permanent skip and starts being what it was meant
to be: a guard for local runs without a server.
Alternatives Considered
- Keep it manual. That is the status quo, and
#143 is what it costs.
- Mock the engine. A mock reproduces the interface, which is the half that already works;
it cannot reproduce SELECT VALUE returning a list of lists.
- Only run it nightly. Cheaper, but a regression then lands on
main first and is found
later. Worth considering if the job proves slow.
Additional Context
Scope: the four files under tests/integration/ are enough to be worth it — the pinning module
alone went from 5 failed, 12 passed to green in #143, and that measurement only existed
because it was run by hand against a real server.
Problem Statement
A suggestion rather than a defect report, and entirely yours to weigh — you know the cost of your
CI minutes and we do not.
CI never starts a SurrealDB instance, so every test that needs one skips. That leaves the only
production backend untested in the pipeline, and it is the reason
#143was necessary: the testthat would have caught those three bugs was added in the same PR that introduced them (
#139,tests/integration/test_surrealdb_pinning.py) and has been skipping ever since.Measured on
v3.0.3:26 test files are gated on
SURREALDB_URL. Thetestjob installs.[dev,server]— not eventhe
surrealdbextra — and runspytest tests/ -m "not stress", so all of them skip green.The parity ratchet added around
#139catches "the method is missing". It cannot catch"the method is present and returns something that looks right" — which is precisely what
#143fixed:SELECT VALUEover an array field returned a list of lists, the unwrap heuristiccollapsed it, and the caller iterated a string character by character. No signature was wrong.
Only a live engine shows that.
Proposed Solution
One integration job with a SurrealDB service container. Everything else stays as it is.
If the official image's entrypoint needs an explicit
startcommand in the Actions servicecontext, the alternative is to run the same container as an ordinary step
(
docker run -d -p 8000:8000 …) and poll/healthbefore the test step — less elegant, but itsidesteps every
options:question above.Use Case
With the job above, that marker stops being a permanent skip and starts being what it was meant
to be: a guard for local runs without a server.
Alternatives Considered
#143is what it costs.it cannot reproduce
SELECT VALUEreturning a list of lists.mainfirst and is foundlater. Worth considering if the job proves slow.
Additional Context
Scope: the four files under
tests/integration/are enough to be worth it — the pinning modulealone went from 5 failed, 12 passed to green in
#143, and that measurement only existedbecause it was run by hand against a real server.