Description
scripts/check_dead_modules.py (added in #137, run in CI as --strict) computes reachability
from the entry points where execution begins — console scripts, __main__ modules, string
literals, and anything an example, benchmark or script imports. Those last ones are roots.
Five of those roots import surreal_memory.storage.sqlite_store, a module #141 removed. The
guard walks the import graph purely from the AST, so it never asks whether an import target
exists. It reports No unreachable modules. and exits 0 while five of the files that define its
reachability set cannot run.
Measured, so the claim is not louder than the evidence: one of the five fails at import
(benchmarks/stress_at_scale.py, whose import is at module level), and the other four fail the
moment the function holding the import is called. All five are dead either way; only one of them
is dead in a way that a plain import would reveal.
#137 is one of the nicer things in this repo and its PR body puts the principle better than we
could:
A guard that reports nothing proves nothing — a broken one behaves identically.
This is that same principle one rung up, and not a flaw in the check's design: the guard itself is
sound, but the roots it trusts are not, and nothing tells it so.
To Reproduce
On a clean v3.0.3 checkout with the package installed:
$ python scripts/check_dead_modules.py
No unreachable modules.
$ echo $?
0
$ python -c "import importlib.util as u; s=u.spec_from_file_location('b','benchmarks/stress_at_scale.py'); m=u.module_from_spec(s); s.loader.exec_module(m)"
ModuleNotFoundError: No module named 'surreal_memory.storage.sqlite_store'
The other four import inside a function, so they load and then raise when that function runs —
rehearsal_coverage_overhead.py on the branch it takes when SURREALDB_URL is unset,
e2e_gemini_recall.py in step 2, and both comparison benchmarks in their nm_setup.
$ grep -rn "SQLiteStorage" benchmarks/ scripts/ | wc -l
18
$ python -c "import surreal_memory.storage as s; print(hasattr(s, 'SQLiteStorage'))"
False
Expected Behavior
Two separable things:
- The guard should not pass while a root imports something that is not there. Resolving each root's
surreal_memory.* imports against the modules actually present in the tree is enough — no
code has to be executed, so the check keeps the "reads the source tree only, no install step"
property #137 deliberately gave it.
- The five files should stop referencing the removed backend.
Actual Behavior
No unreachable modules., exit 0, in CI (.github/workflows/ci.yml:37) and locally
(make audit-dead).
Code Sample
The 18 references, all of them real imports and uses — not prose:
benchmarks/stress_at_scale.py:33 from …sqlite_store import SQLiteStorage ← module level
benchmarks/stress_at_scale.py:136,180,214,232 type annotations
benchmarks/stress_at_scale.py:294 SQLiteStorage(db_path=str(db_path))
benchmarks/stress_at_scale.py:379,482 report text: "Real SQLiteStorage … WAL mode"
benchmarks/rehearsal_coverage_overhead.py:117,121 the else branch taken when SURREALDB_URL is unset
scripts/benchmark_mem0_vs_nm.py:121,286,294
scripts/benchmark_cognee_vs_nm.py:116,276,284
scripts/e2e_gemini_recall.py:63,66
benchmarks/stress_at_scale.py is the only one with a module-level import, so it is the only
one an import test catches; the other four fail at call time. For contrast, the 16 occurrences
of the same name
under src/ and tests/ are all historical prose in docstrings ("used to reach into a private
SQLiteStorage attribute") — those are accurate and should stay.
Error Message
ModuleNotFoundError: No module named 'surreal_memory.storage.sqlite_store'
Environment
- OS: Linux (x86-64)
- Python version: 3.12.13
- Surreal-Memory version: 3.0.3 (
ac20df41)
- Installation method: source,
pip install -e ".[dev,server,surrealdb]"
Additional Context
We did not open a pull request for the second half, because the fix is not mechanical and the
choice is yours. The symbol is used, not merely imported — as a constructor and in type
annotations — so deleting the import does not leave working code. The options we can see:
- Delete.
benchmarks/stress_at_scale.py is SQLite-specific end to end (its own report says
"Real SQLiteStorage benchmarks", "aiosqlite, WAL mode"), so with the backend gone it measures
something that no longer ships.
- Port.
#140 made InMemoryStorage implement the full interface, which is a drop-in for
scripts/e2e_gemini_recall.py and the two comparison benchmarks — but for a storage
benchmark it changes what is being measured, and the prose claims about WAL mode would have to
go with it.
- Require SurrealDB.
benchmarks/rehearsal_coverage_overhead.py already calls the SurrealDB
path "the production-accurate path" in its own message; dropping the fallback branch is the
smallest change there.
We are happy to send whichever of these you prefer as a PR, including the guard change.
Related: #137 (the guard), #141 (backend removal), #140 (InMemoryStorage completeness).
Description
scripts/check_dead_modules.py(added in#137, run in CI as--strict) computes reachabilityfrom the entry points where execution begins — console scripts,
__main__modules, stringliterals, and anything an example, benchmark or script imports. Those last ones are roots.
Five of those roots import
surreal_memory.storage.sqlite_store, a module#141removed. Theguard walks the import graph purely from the AST, so it never asks whether an import target
exists. It reports
No unreachable modules.and exits 0 while five of the files that define itsreachability set cannot run.
Measured, so the claim is not louder than the evidence: one of the five fails at import
(
benchmarks/stress_at_scale.py, whose import is at module level), and the other four fail themoment the function holding the import is called. All five are dead either way; only one of them
is dead in a way that a plain
importwould reveal.#137is one of the nicer things in this repo and its PR body puts the principle better than wecould:
This is that same principle one rung up, and not a flaw in the check's design: the guard itself is
sound, but the roots it trusts are not, and nothing tells it so.
To Reproduce
On a clean
v3.0.3checkout with the package installed:The other four import inside a function, so they load and then raise when that function runs —
rehearsal_coverage_overhead.pyon the branch it takes whenSURREALDB_URLis unset,e2e_gemini_recall.pyin step 2, and both comparison benchmarks in theirnm_setup.Expected Behavior
Two separable things:
surreal_memory.*imports against the modules actually present in the tree is enough — nocode has to be executed, so the check keeps the "reads the source tree only, no install step"
property
#137deliberately gave it.Actual Behavior
No unreachable modules., exit 0, in CI (.github/workflows/ci.yml:37) and locally(
make audit-dead).Code Sample
The 18 references, all of them real imports and uses — not prose:
benchmarks/stress_at_scale.pyis the only one with a module-level import, so it is the onlyone an import test catches; the other four fail at call time. For contrast, the 16 occurrences
of the same name
under
src/andtests/are all historical prose in docstrings ("used to reach into a privateSQLiteStorageattribute") — those are accurate and should stay.Error Message
Environment
ac20df41)pip install -e ".[dev,server,surrealdb]"Additional Context
We did not open a pull request for the second half, because the fix is not mechanical and the
choice is yours. The symbol is used, not merely imported — as a constructor and in type
annotations — so deleting the import does not leave working code. The options we can see:
benchmarks/stress_at_scale.pyis SQLite-specific end to end (its own report says"Real SQLiteStorage benchmarks", "aiosqlite, WAL mode"), so with the backend gone it measures
something that no longer ships.
#140madeInMemoryStorageimplement the full interface, which is a drop-in forscripts/e2e_gemini_recall.pyand the two comparison benchmarks — but for a storagebenchmark it changes what is being measured, and the prose claims about WAL mode would have to
go with it.
benchmarks/rehearsal_coverage_overhead.pyalready calls the SurrealDBpath "the production-accurate path" in its own message; dropping the fallback branch is the
smallest change there.
We are happy to send whichever of these you prefer as a PR, including the guard change.
Related:
#137(the guard),#141(backend removal),#140(InMemoryStoragecompleteness).