fix: support untyped NULL input for median - #24104
Conversation
|
Hi, this is my first contribution to DataFusion. Could a committer please trigger CI when convenient? Thanks! |
| ], | ||
| Volatility::Immutable, | ||
| ), | ||
| signature: Signature::user_defined(Volatility::Immutable), |
There was a problem hiding this comment.
it would be preferable to keep the old signature if possible, as we prefer avoiding user defined unless strictly necessary
could we fix the physical execution path to account for null type input?
There was a problem hiding this comment.
Thanks for the suggestion. I restored the original declarative signature and moved the Null handling to the physical execution path.
Null input now uses the existing NoopAccumulator pattern, has a matching Null state field, and falls back to the generic groups accumulator adapter. I also updated the regression test to expect NULL with DataType::Null.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #24104 +/- ##
==========================================
- Coverage 81.03% 81.02% -0.01%
==========================================
Files 1105 1105
Lines 379846 379859 +13
Branches 379846 379859 +13
==========================================
- Hits 307793 307790 -3
- Misses 53831 53841 +10
- Partials 18222 18228 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Which issue does this PR close?
Rationale for this change
median(NULL)worked in DataFusion 53 but regressed in DataFusion 54. The untyped NULL argument remainedDataType::Null, causing physical planning to create a median accumulator for an unsupported input type.What changes are included in this PR?
median.DataType::Nullin physical execution usingNoopAccumulator.median(NULL).Are these changes tested?
Yes.
cargo fmt --all -- --checkcargo test --profile=ci --test sqllogictests -- aggregate.sltcargo test -p datafusion-functions-aggregatecargo clippy -p datafusion-functions-aggregate --all-targets --all-features -- -D warningsAre there any user-facing changes?
Yes.
median(NULL)now returns NULL withDataType::Nullinstead of failing during physical execution.