[SPARK-58736][SQL][PYTHON][CONNECT] Support incremental Python aggregators via Arrow - #57952

Open

HyukjinKwon wants to merge 12 commits into

apache:masterfrom

HyukjinKwon:SPARK-python-arrow-incremental-aggregator

Commits on Aug 12, 2026

[WIP][SQL][PYTHON] Support incremental Python aggregators via Arrow
HyukjinKwon
committed
abadf10
[SQL][PYTHON][CONNECT] Support incremental Python aggregators over Spark Connect
HyukjinKwon
committed
b8310d5
[SQL][PYTHON] Rename arrow_udaf to udaf and require PyArrow
HyukjinKwon
committed
03083b9
[SQL][PYTHON] Move aggregator module to pyspark.sql and set version to 4.4.0
HyukjinKwon
committed
8b0448b
[SQL][PYTHON][CONNECT] Support spark.udf.register for the incremental aggregator
HyukjinKwon
committed
3e2bb59
[SQL][PYTHON] Fix custom-error lint and test import
HyukjinKwon
committed
19a53da
[SQL][PYTHON] Address review: stream partial batches and fix empty global aggregation
HyukjinKwon
committed
a12f825
[SQL][PYTHON] Clarify mixed-UDF error and test multiple aggregators
HyukjinKwon
committed
3917089

Commits on Aug 13, 2026