fix: address 6 open issues (#64, #150-#154) and bump to v3.3.0

.github/workflows/ci.yml

@@ -138,6 +138,63 @@ jobs:
       - name: Run security rules
         run: ruff check src/ --select S --ignore S101,S110,S112,S311,S324
 
+  integration:
+    name: Integration (SurrealDB)
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    env:
+      SURREALDB_URL: http://127.0.0.1:8001
+      SURREALDB_USER: root
+      SURREALDB_PASS: root
+      SURREALDB_NS: smem_ci
+      SURREALDB_DB: smem_ci
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      # `services:` can't run this: the image's ENTRYPOINT is `/surreal` with no
+      # default CMD (no command == prints help and exits), and `services:` only
+      # accepts `options:` (docker-create flags), never a container command. A
+      # plain `docker run` step is the only way to pass the required `start …`
+      # arguments. Command mirrors docker-compose.surrealdb.yml, minus the
+      # persistence path: an in-memory datastore is enough for a run that's
+      # thrown away afterward and needs no root-owned named volume.
+      - name: Start SurrealDB
+        run: |
+          docker run -d --name surrealdb-ci -p 8001:8001 \
+            surrealdb/surrealdb:v3.2.0 \
+            start memory --user root --pass root --bind 0.0.0.0:8001 \
+            --allow-experimental gql --allow-eval-query
+
+      - name: Wait for SurrealDB to be ready
+        run: |
+          for _ in $(seq 1 30); do
+            if docker exec surrealdb-ci /surreal isready --endpoint http://localhost:8001; then
+              exit 0
+            fi
+            sleep 1
+          done
+          echo "SurrealDB did not become ready in time"
+          docker logs surrealdb-ci
+          exit 1
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,server,surrealdb]"
+
+      # No -n auto here: parallel workers hammering one shared SurrealDB
+      # connection produce connection resets under load unrelated to any real
+      # regression. Correctness, not speed, is the point of this job.
+      - name: Run tests against a live SurrealDB
+        run: pytest tests/ -v --timeout=120 -m "not stress"
+
+      - name: SurrealDB logs (on failure)
+        if: failure()
+        run: docker logs surrealdb-ci
+
   build:
     name: Build Package
     runs-on: ubuntu-latest

CHANGELOG.md

@@ -5,6 +5,92 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [3.3.0] — 2026-08-03 — a contributor's stop-word acronym fix, and a sweep of small honesty bugs
+
+### Added
+
+- **A CI job now runs the full test suite against a live SurrealDB.** Every prior job
+  either mocked the database or skipped whenever `SURREALDB_URL` was unset, so the only
+  production backend went untested in the pipeline — the exact gap `#143`'s bug walked
+  through (a query that returns plausible garbage instead of failing loudly). The job
+  starts the official image via a `docker run` step (GitHub Actions' `services:` cannot
+  pass the datastore/flags SurrealDB's entrypoint requires), waits for `/surreal isready`,
+  then runs the whole suite sequentially — no `-n auto`, since parallel workers sharing one
+  connection produce transaction-conflict errors unrelated to any real regression.
+
+### Fixed
+
+- **Polish stop words `ma`/`na`/`co`/`sa` swallowed the acronyms `MA`/`NA`/`CO`/`SA`.**
+  A contributor's audit (`#64`) showed the fix everyone assumed — removing the words from
+  the stop list — was wrong: `N/A` and `S.A.` never actually collide (punctuation
+  fragments them below the minimum word length before the stop-word check ever runs), so
+  removing the words would only trade real function-word noise for a niche acronym gain.
+  A token that is ALL-CAPS in the source now survives even when its lowercased form is a
+  stop word, gated by how much of the text is uppercase — a shouted note or an all-caps
+  heading has no acronym to rescue, only every word incidentally capitalized, and must not
+  have its stop-word filtering disabled wholesale.
+- **`check_dead_modules` passed silently while five files imported a module that no
+  longer exists.** Reachability is computed from each file's imports, so one naming a
+  module the tree no longer has (the SQLite backend, removed in `3.0.0`) contributed
+  nothing to the graph and produced no diagnostic — the guard reported `No unreachable
+  modules.` while five benchmark/script files could not run. It now resolves every import
+  against the tree and reports the ones that don't. The five files were fixed: one
+  (`stress_at_scale.py`) measured only the removed backend and is deleted; the other four
+  now require a live `SURREALDB_URL` instead of silently falling back to it.
+- **Two read-only endpoints in the sync hub could redirect scheduled maintenance onto
+  the wrong brain.** `GET /hub/status/{id}` and `GET /hub/devices/{id}` switched the
+  process-wide shared storage's active brain to answer a lookup, and the background
+  consolidation/decay loops read that same mutable state on their next tick — so a
+  read-only request for brain B could cause the next scheduled pass to run against B
+  instead of the brain the operator actually left active. Both endpoints now read through
+  an isolated, brain-scoped connection instead of mutating the shared one — the same
+  pattern already used for reasoning-training's read endpoints.
+- **The tool-stats `days` filter changed the daily chart but not the summary above it.**
+  `get_tool_stats` took no `days` argument, so its summary was always computed over all
+  time; switching the dashboard's range selector between 7/30/90 days produced a
+  byte-identical summary while the chart below it changed correctly. The summary now
+  respects the same window as the daily series, and the two storage methods behind it are
+  declared on the storage interface (with an in-memory implementation) instead of being
+  reachable only on the SurrealDB backend behind an unchecked attribute access.
+- **A `SELECT VALUE` query on an array-typed field could be read as if it were rows.**
+  The shared query helper's return type says it hands back row dictionaries, which is not
+  what a `SELECT VALUE` query returns — the field it selects for one matching row can
+  itself be an array, indistinguishable by shape alone from several separate scalar rows.
+  That mismatch is the mechanism behind a prior release's fixed bug (a result iterated
+  character by character). The one live call site reachable this way now goes through a
+  separate, honestly-typed helper instead of the row-shaped one.
+- **The Settings brain-files panel showed a plausible path to a file that was never
+  written.** A brain that exists only in SurrealDB has no on-disk database file, but the
+  panel built and returned a path for it regardless — worse when a stale file from an
+  older install happened to sit at that exact path for one brain while its neighbours
+  showed nothing. The path is now omitted rather than fabricated when the file does not
+  exist.
+
+### Removed
+
+- **The `smem_drift` tool and its underlying tag-drift detector.** Every call path into it
+  swallowed its own failures, so on the only shipped backend it always reported "clean" —
+  indistinguishable from a real analysis that found nothing, because the two things it
+  measures (tag co-occurrence, session summaries) were never implemented on that backend
+  and never had been, even before the SQLite backend's removal. Its `merge` action never
+  actually merged tags either. `TagNormalizer`'s own drift detection, which needs no
+  storage and already backs `smem doctor`'s tag-drift warnings, remains and covers the
+  same practical need. A small unrelated write path — recording tag co-occurrence for the
+  now-removed detector, and a periodic session-summary persist with the same swallowed
+  failure — is removed alongside it.
+- **A CLI sandbox-guard helper was renamed** from `ensure_aiosqlite_or_exit_cli` to
+  `ensure_sqlite_or_exit_cli`: it has only ever checked the stdlib `sqlite3` module, never
+  the optional `aiosqlite` package, and the name was misleading anyone reading the CLI
+  startup path. Internal-only; no public interface changed.
+
+### Changed
+
+- **BREAKING:** the `smem_drift` MCP tool is gone. If anything called it expecting a real
+  answer, that answer was never trustworthy on the SurrealDB backend to begin with — see
+  Removed, above.
+
+**Full diff**: https://github.com/acidkill/surreal-memory/compare/v3.2.0...v3.3.0
+
 ## [3.2.0] — 2026-08-03 — reasoning mining sees every profile; embeddings stop inheriting a stranger's endpoint
 
 ### Added

README.md

@@ -110,7 +110,7 @@ smem recall "auth bug"
 
 ## 3 Tools. That's It.
 
-58 MCP tools are available, but you only need three:
+57 MCP tools are available, but you only need three:
 
 | Tool | What it does |
 |------|-------------|
@@ -126,7 +126,7 @@ Everything else — sessions, context loading, habit tracking, maintenance — w
 
 ```
                     ┌──────────────────────────────┐
-                    │       MCP Server (58 tools)   │
+                    │       MCP Server (57 tools)   │
                     └──────────┬───────────────────┘
                                │
                     ┌──────────▼───────────────────┐

benchmarks/rehearsal_coverage_overhead.py

@@ -11,15 +11,14 @@
 actual ``get_maturation``/``save_maturation`` round trips, since that IS the
 added cost: each extra rehearsed fiber is one more read plus one more write.
 
-Backend: SurrealDB when ``SURREALDB_URL`` is set (matches this repo's
-live-test gating convention), falling back to a temporary SQLite file
-otherwise. This matters: SQLite's ``find_fibers`` goes through an indexed
-``fiber_neurons`` junction table, while the SurrealDB backend's equivalent
-query is an unindexed array-containment scan over ``fiber.neuron_ids`` (no
-index on that field, confirmed via an independent database-review pass on
-this run) -- an earlier SQLite-only run of this benchmark was measuring a
-different, faster query shape than what production actually pays. Prefer the
-SurrealDB path whenever a live instance is available.
+Backend: SurrealDB, required -- ``SURREALDB_URL`` must be set (matching this
+repo's live-test gating convention), and without it the benchmark exits
+instead of measuring something else. The old fallback to a temporary SQLite
+file was misleading before v3.0.0 deleted that backend: SQLite's
+``find_fibers`` went through an indexed ``fiber_neurons`` junction table,
+while the SurrealDB equivalent is an unindexed array-containment scan over
+``fiber.neuron_ids``. The fallback was measuring a different, faster query
+shape than production pays.
 
 Intentionally not a CI assertion (microbenchmark timing is too noisy for a
 hard threshold).
@@ -85,45 +84,40 @@ async def _run_against(storage: Any, brain_id: str) -> tuple[float, float]:
 
 async def _main() -> None:
     surrealdb_url = os.environ.get("SURREALDB_URL")
+    if not surrealdb_url:
+        print("SURREALDB_URL is not set.")
+        print()
+        print("This benchmark needs the production backend: the cost it measures is")
+        print("get_maturation/save_maturation round trips, which only a live engine")
+        print("charges honestly. Start SurrealDB and set SURREALDB_URL, e.g.")
+        print("    docker compose -f docker-compose.surrealdb.yml up -d")
+        print("    SURREALDB_URL=ws://localhost:8001/rpc python benchmarks/rehearsal_coverage_overhead.py")
+        raise SystemExit(1)
+
+    backend = "SurrealDB"
     brain_id = f"bench-rehearsal-{uuid4().hex[:8]}"
 
-    if surrealdb_url:
-        backend = "SurrealDB"
-        from surreal_memory.storage.surrealdb.store import SurrealDBStorage
-
-        storage = SurrealDBStorage(url=surrealdb_url)
-        await storage.initialize()
-        try:
-            old_ms, new_ms = await _run_against(storage, brain_id)
-        finally:
-            # Cleanup discipline (project hard rule): delete only this run's
-            # own brain, by its exact id, never `default`. Re-fetch the real
-            # RecordID via SELECT rather than reconstructing one from the
-            # string id -- a hand-built `type::record('brain', $bid)` looked
-            # like it succeeded (no error) but silently matched zero rows,
-            # confirmed live: an earlier version of this script left its test
-            # brain orphaned this way even though the query "succeeded".
-            await storage.clear(brain_id)
-            rows = await storage._query(
-                "SELECT id FROM brain WHERE id = type::record('brain', $bid)", bid=brain_id
-            )
-            for row in rows:
-                await storage._query("DELETE $rid", rid=row["id"])
-            await storage.close()
-    else:
-        backend = (
-            "SQLite (SurrealDB not reachable -- set SURREALDB_URL for the production-accurate path)"
+    from surreal_memory.storage.surrealdb.store import SurrealDBStorage
+
+    storage = SurrealDBStorage(url=surrealdb_url)
+    await storage.initialize()
+    try:
+        old_ms, new_ms = await _run_against(storage, brain_id)
+    finally:
+        # Cleanup discipline (project hard rule): delete only this run's
+        # own brain, by its exact id, never `default`. Re-fetch the real
+        # RecordID via SELECT rather than reconstructing one from the
+        # string id -- a hand-built `type::record('brain', $bid)` looked
+        # like it succeeded (no error) but silently matched zero rows,
+        # confirmed live: an earlier version of this script left its test
+        # brain orphaned this way even though the query "succeeded".
+        await storage.clear(brain_id)
+        rows = await storage._query(
+            "SELECT id FROM brain WHERE id = type::record('brain', $bid)", bid=brain_id
         )
-        from surreal_memory.storage.sqlite_store import SQLiteStorage
-
-        with tempfile.TemporaryDirectory() as tmp:
-            db_path = Path(tmp) / "bench.db"
-            storage = SQLiteStorage(db_path)
-            await storage.initialize()
-            try:
-                old_ms, new_ms = await _run_against(storage, brain_id)
-            finally:
-                await storage.close()
+        for row in rows:
+            await storage._query("DELETE $rid", rid=row["id"])
+        await storage.close()
 
     print(f"backend:                                  {backend}")
     print(f"reinforce() at limit=10 (old default):   {old_ms:8.2f} ms/call")