[SPARK-57091][SQL] Add BroadcastNearestByJoinExec to avoid cross-product materialization - #56101

Open

yadavay-amzn wants to merge 16 commits into

apache:masterfrom

yadavay-amzn:proto/nearestby-streaming-heap

Commits on Aug 10, 2026

Add NearestByJoin streaming heap with memory benchmark
yadavay-amzn
committed
eb79470
[SPARK-57091][SQL][FOLLOWUP] Declare NOT_APPLICABLE binding policy for spark.sql.join.nearestBy.broadcast.enabled
yadavay-amzn
committed
71ae041
[SPARK-57091][SQL][FOLLOWUP] Use 4.3.0 conf version and copy ranking row only for variable-length types
yadavay-amzn
committed
f022124
[SPARK-57091][SQL] Initialize non-deterministic ranking projection and fix SQLConf scaladoc link
yadavay-amzn
committed
34c7d2a
[SPARK-57091][SQL] Make NearestByJoin broadcast rewrite stats-free and defer sizing to the planner
yadavay-amzn
committed
ede05b3
[SPARK-57091][SQL] Address review: contract docs, remove dead canBroadcastRight, crossJoin divergence, EXPLAIN output, heap sizing, parity + eviction tests
yadavay-amzn
committed
bd0e765
[SPARK-57091][SQL] Address review: remove stray blank line in SparkStrategies
yadavay-amzn
committed
60c7730
[SPARK-57091][SQL] Review: hoist all-null right row, fix test comment, resolve scaladoc links, clean test headings
yadavay-amzn
committed
15bd189
[SPARK-57091][SQL] Review: use InternalRow.copyValue for retained variable-width ranking values
yadavay-amzn
committed
cbbab42
[SPARK-57091][SQL] Review: version 4.4.0, AQE reOptimize coverage, outputOrdering, EXPLAIN field label, rule conf
yadavay-amzn
committed
cded42f
[SPARK-57091][SQL] Route two-sided Python UDF rankings through the rewrite; reuse HeapEntry to bound allocations
yadavay-amzn
committed
3e6e08a

Commits on Aug 12, 2026