fix(helm): verify mongo restore completeness and allow skipping collections

[yodem]

Follow-up to Sefaria/cauldrons#152. That PR stopped a timed-out restore from reporting success; this one stops an incomplete restore from going unnoticed at all, and gives cauldrons a way to skip the expensive half of the dump.

Independent of #152 and safe to merge in either order.

1. restore.verify (default true)

Nothing in the pipeline ever checked that mongorestore produced a complete database, which is why a cauldron with 8 of 73 collections looked identical to a healthy one. After the restore, the job now compares the collections extracted from the dump against what mongo actually holds:

ERROR: restore is incomplete. Expected 73 collections, found 8.
Missing collections:
  - apikeys
  - arukh_hashulchan
  ...

Self-calibrating — the expected set is derived from the .bson files on disk, so excluded collections are not expected and no collection list is hardcoded anywhere.

Scope limit worth knowing: this catches an incomplete restore that finishes. A restore killed mid-flight by the Helm hook timeout never reaches the check — that path is what install.timeout / install.remediation in cauldrons#152 handles. The two are complementary, which is why both exist.

2. restore.excludeCollections (default [])

restore:
  excludeCollections: [sheets]

Exclusion happens at tar-extraction time, not restore time, so skipped bytes are never written to local disk. That matters because the untar is the disk-bound stage and the slowest part of a restore — measured at 13 m 38 s of a 30 m budget, running at 37 MB/s against a node disk at 98% utilization. So excluding a collection saves time twice.

sheets is ~14.2 GB of the ~27 GB dump. For cauldrons that never exercise user sheets, this roughly halves the restore.

Verification

Template renders and the generated bash passes bash -n in all three configurations (default / with exclusions / verify off). The exclude and detection logic was functionally tested against a fake dump built with the real ./dump/<db>/<coll>.bson layout:

case	result
exclusions removed from extraction	✅ only non-excluded .bson present
complete restore	✅ passes
truncated restore	✅ fails, names the missing collections
mongo has extra collections (views)	✅ passes, no false positive

helm lint clean; full-chart helm template exits 0.

Not changed here

The [[ ! -z "MONGO_REPLICASET_NAME" ]] test just above the edited region is missing its $, so it is always true and appends an empty &replicaSet= to the URI. Restores work today, so correcting it would change connection-string behaviour for every environment — left for its own change rather than smuggled into this one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HFb3rAtzXJ4Ue7w4G3c4VW

[gitvelocity-reviewer]

📊 Code Quality Score: 7/100

28 × 0.25 (Micro ESF) = 7

Category	Score	Factors
🔭 Scope	5/20	Two files modified: helm-chart/sefaria/templates/configmap/mongo-restore.yaml and helm-chart/sefaria/values.yaml. Both are within the same Helm chart subsystem. Two new values.yaml keys (excludeCollections, verify) are introduced as the public interface.
🏗️ Architecture	3/20	No new module boundary or dependency introduced. The change adds a Helm range block and an if block inside an existing configmap template, keeping all logic within the existing restore script structure.
⚙️ Implementation	8/20	The exclusion block builds a bash array with the TAR_EXCLUDES+=() pattern and uses the ${TAR_EXCLUDES[@]+"${TAR_EXCLUDES[@]}"} safe-expansion idiom to avoid unbound-variable errors on an empty array. The verification block uses comm -23 on two sorted streams — one from ls on extracted .bson files, one from mongo --eval getCollectionNames() — to compute the set difference and exits non-zero with a formatted missing-collection report. set -o pipefail is added at the top of the script.
⚠️ Risk	7/20	The restore job is a recovery-critical path: a bug in the exclusion logic could silently drop collections from the restore, and a bug in the verification logic could reject a valid restore or pass a truncated one. verify: true is the default, so the new verification behavior is active for all existing deployments on upgrade. The mongo CLI used in the verification block is removed in MongoDB 6.0, which would cause the verification step to fail on newer clusters.
✅ Quality	3/15	No automated tests accompany the change. Inline comments explain the disk-bound rationale for extraction-time exclusion and the excluded-collections-not-expected logic in the verification block. No test drives the Helm template rendering or the bash script paths.
🔒 Perf / Security	2/5	The comment in values.yaml quantifies the performance motivation: sheets alone is ~14GB of the ~27GB dump, so excluding at tar extraction time avoids writing that data to disk entirely. No security considerations are present in this change.

Was this score accurate? 👍 Yes · 👎 No

How this was scored →

_{Scored by GitVelocity · How are scores calculated?}