Skip to content

Mark scalar subquery results nullable - #64

Merged
osipovartem merged 1 commit into
embucket-sync-df55.0.0from
scalar-subquery-nullability
Sep 5, 2026
Merged

Mark scalar subquery results nullable#64
osipovartem merged 1 commit into
embucket-sync-df55.0.0from
scalar-subquery-nullability

Conversation

@osipovartem

@osipovartem osipovartem commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • mark scalar subquery expressions nullable because SQL returns NULL when the subquery produces zero rows
  • preserve the inner field type and metadata while forcing nullable output schema
  • cover both ExprSchemable::nullable and ExprSchemable::to_field with a focused unit test

Why

Rustice's live Snowflake compatibility oracle exposed a valid scalar subquery over an empty result. Execution produced NULL, but DataFusion propagated the non-nullable inner field and Arrow rejected the result with:

Column 'scalar_empty' is declared as non-nullable but contains null values

Validation

  • cargo +1.97.0 test -p datafusion-expr -q (249 unit tests and 55 doc/integration tests passed; 3 ignored)
  • cargo +1.97.0 clippy -p datafusion-expr --all-targets -- -D warnings
  • cargo +1.97.0 fmt --all -- --check

@osipovartem
osipovartem merged commit be9b970 into embucket-sync-df55.0.0 Sep 5, 2026
20 checks passed
@osipovartem
osipovartem deleted the scalar-subquery-nullability branch September 5, 2026 07:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant