Branch merge scales with the full target table, not the branch delta — OOMs / exhausts the Lance memory pool on a large Vector column

[FlorianSeemann]

Summary

Merging a branch (POST /branches/merge, and also a local bucket-direct load --mode merge) into a target that holds a large Vector column has a cost that scales with the size of the target table, not with the size of the branch's delta. Merging a branch that adds only ~50 rows to a ~750k-row table of Vector(3072) fails the same way as a large branch. On a memory-constrained host this manifests two ways:

• Server (Railway, 8 GB host): the merge OOM-crashes the server process.

• Local bucket-direct CLI: the merge fails deterministically with

  LanceError(IO): Resources exhausted: Failed to allocate 119.5 MB for HashJoinInput —
  100 MB remain available for the total pool
  

The HashJoinInput allocation and the "100 MB total pool" strongly suggest the merge reconciliation hash-joins the branch against the entire target table's columns (including the big embedding column) rather than reconciling only the rows the branch changed.

Environment

• Engine v0.8.1, Lance-backed storage (self-hosted MinIO, s3://…/graphs/<g>.omni).
• Table: Chunk with embedding: Vector(3072) (float32 ≈ 12 KB/row raw).
• Row count / size where it breaks: ~751k rows ≈ 9.2 GB of raw vector payload.
• Prod host RAM: 8 GB.

Reproduction

1. Have a table with a Vector(3072) column and enough rows that the column materializes to more than host RAM / the Lance memory pool (here ~9.2 GB at 751k rows).
2. Create a branch that inserts a small number of new rows (e.g. 50).
3. Merge the branch into the parent.

Expected: merge cost is proportional to the branch delta (~50 rows).
Actual: merge tries to allocate against the full target column → OOM (server) / Resources exhausted: … HashJoinInput … 100 MB … pool (local CLI).

Evidence it's target-size-driven, not delta-driven

• The same merge succeeded earlier when the table was ~304k rows (≈3.7 GB, fit in 8 GB) and consistently fails now at ~751k rows (≈9.2 GB). Only the target size changed.
• Batch size is irrelevant: reducing the branch to ~50 rows (and, locally, --chunk-size 10) does not help — the failure recurs, because the cost is on the target side of the join.

Workaround we're using

• Raising LANCE_MEM_POOL_SIZE (e.g. 4294967296 = 4 GiB) before the CLI runs lets the local bucket-direct merge succeed — but this only defers the ceiling; the pool still has to hold a full-target-column join, so it scales with the table, not the delta.
• We've moved all Vector-column writes off the server merge path entirely (bucket-direct, single-writer) as a stopgap.

Ask

Make branch merge reconcile only the branch delta (the rows added/changed on the branch) rather than hash-joining / materializing the full target table's columns. That would make merge cost proportional to the change set and remove the table-size ceiling for both the server and CLI paths. If a full-table join is fundamental to the current reconciliation algorithm, is there a way to (a) exclude large Vector columns from the join keys, or (b) stream/spill the join so it stays within a bounded pool regardless of target size?

Happy to provide Lance table stats, a minimal repro, or full logs.