Skip to content

Physical planning: cast low-medium cardinality columns to dictionary arrays before aggregation #22891

Description

@Rich-T-kid

Is your feature request related to a problem or challenge?

GroupValuesDictionary can provide up to a 2-5x speedup over GroupValuesRows for aggregations, but it is only used when the input data is already a dictionary-encoded column.

Describe the solution you'd like

A physical optimizer rule could inspect column statistics at plan time and cast low-medium cardinality Utf8 columns to Dictionary(Int32, Utf8) before aggregation, allowing GroupValuesDictionary to be selected automatically without any changes from the user.
Parquet files keep metadata about cardinality, this may be a good place to start

Describe alternatives you've considered

No response

Additional context

#21765

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions