perf: Concretely typed TopK ArrowHashTable storage by MassivePizza · Pull Request #23609 · apache/datafusion

MassivePizza · 2026-07-15T15:13:27Z

Which issue does this PR close?

N/A

Rationale for this change

Eliminate the overhead of casting with dyn Any for every item in ArrowHashTable.

What changes are included in this PR?

Cast once per batch and store the typed array.
PrimitiveHashTables was already generic, so this was a big and simple win.
StringHashTable was made ~~to use an enum for consistency/readability~~ generic for a miniscule perf improvement over enum.

Are these changes tested?

Should be covered by existing tests.
For benching try cargo bench -p datafusion --bench topk_aggregate -- "top k=10 aggregate".

top k=10 aggregate 10000000 time-series rows
                        time:   [10.787 ms 10.940 ms 11.093 ms]
                        change: [−38.879% −36.987% −35.111%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

top k=10 aggregate 10000000 worst-case rows
                        time:   [17.414 ms 17.797 ms 18.203 ms]
                        change: [−46.131% −44.095% −41.958%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

Benchmarking top k=10 aggregate 10000000 time-series rows [Utf8View]: Collecting 100 samples in estimated 5.9278 s (500 itera
top k=10 aggregate 10000000 time-series rows [Utf8View]
                        time:   [11.777 ms 11.954 ms 12.147 ms]
                        change: [−37.279% −35.443% −33.760%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe

Benchmarking top k=10 aggregate 10000000 worst-case rows [Utf8View]: Collecting 100 samples in estimated 6.3472 s (300 iterat
top k=10 aggregate 10000000 worst-case rows [Utf8View]
                        time:   [20.337 ms 20.786 ms 21.301 ms]
                        change: [−37.920% −35.713% −33.401%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

Are there any user-facing changes?

None.

alamb · 2026-07-16T17:26:44Z

run benchmark topk_aggregate

adriangbot · 2026-07-16T17:27:53Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4994752917-1116-k6xvg 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (8a7b678) to 95de385 (merge-base) diff using: topk_aggregate
Results will be posted here when complete

File an issue against this benchmark runner

codecov-commenter · 2026-07-16T17:51:47Z

Codecov Report

❌ Patch coverage is 96.82540% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.29%. Comparing base (9b3b518) to head (b3f9734).

Files with missing lines	Patch %	Lines
...on/physical-plan/src/aggregates/topk/hash_table.rs	96.15%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #23609   +/-   ##
=======================================
  Coverage   81.29%   81.29%           
=======================================
  Files        1110     1110           
  Lines      385336   385311   -25     
  Branches   385336   385311   -25     
=======================================
- Hits       313261   313257    -4     
+ Misses      53594    53569   -25     
- Partials    18481    18485    +4

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

adriangbot · 2026-07-16T18:05:44Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     63.7±6.94ms        ? ?/sec    1.00     63.5±6.93ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.01     65.1±8.58ms        ? ?/sec    1.00     64.3±8.09ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.41      5.8±0.14ms        ? ?/sec    1.00      4.1±0.07ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.05     55.7±3.17ms        ? ?/sec    1.00     53.0±3.16ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.41      5.8±0.08ms        ? ?/sec    1.00      4.1±0.06ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.07     56.6±2.91ms        ? ?/sec    1.00     53.0±2.95ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.00     51.1±2.32ms        ? ?/sec    1.06     54.2±2.90ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.00     49.1±1.39ms        ? ?/sec    1.06     52.0±2.38ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   299.2±16.08ms        ? ?/sec    1.04   311.5±17.93ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   289.7±10.10ms        ? ?/sec    1.07   310.2±11.00ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.30     11.7±0.35ms        ? ?/sec    1.00      9.0±0.24ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.26     13.0±0.47ms        ? ?/sec    1.00     10.4±0.34ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.29     18.2±0.97ms        ? ?/sec    1.00     14.2±0.73ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.27     19.8±1.15ms        ? ?/sec    1.00     15.6±0.77ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.01     11.6±0.25ms        ? ?/sec    1.00     11.5±0.21ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.00      9.8±0.18ms        ? ?/sec    1.00      9.8±0.17ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.6±0.28ms        ? ?/sec    1.01     11.7±0.41ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.8±0.26ms        ? ?/sec    1.00      9.8±0.21ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1140.3s
Peak memory	2.1 GiB
Avg memory	323.3 MiB
CPU user	2089.2s
CPU sys	118.7s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1120.2s
Peak memory	2.1 GiB
Avg memory	319.2 MiB
CPU user	1979.1s
CPU sys	126.4s
Peak spill	0 B

File an issue against this benchmark runner

alamb · 2026-07-16T18:10:53Z

Benchmarks look good -- I am running them once more to see if htey are reproducable

alamb · 2026-07-16T18:10:54Z

run benchmark topk_aggregate

adriangbot · 2026-07-16T18:13:38Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c4995135144-1119-ltwqj 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (8a7b678) to 95de385 (merge-base) diff using: topk_aggregate
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-07-16T18:40:05Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     61.5±8.15ms        ? ?/sec    1.06     65.1±6.29ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     59.5±6.85ms        ? ?/sec    1.03     61.5±5.50ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.40      5.8±0.08ms        ? ?/sec    1.00      4.1±0.07ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.00     31.8±0.48ms        ? ?/sec    1.65     52.4±2.91ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.39      5.8±0.06ms        ? ?/sec    1.00      4.1±0.06ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     31.8±0.30ms        ? ?/sec    1.70     53.9±2.74ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.01     52.3±2.59ms        ? ?/sec    1.00     51.9±2.41ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.01     50.2±2.42ms        ? ?/sec    1.00     49.7±1.68ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.08   323.4±16.09ms        ? ?/sec    1.00   300.1±13.64ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   299.5±11.02ms        ? ?/sec    1.00   299.2±10.60ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.31     11.7±0.42ms        ? ?/sec    1.00      9.0±0.30ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.26     13.0±0.46ms        ? ?/sec    1.00     10.3±0.35ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.29     18.2±1.02ms        ? ?/sec    1.00     14.1±0.71ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.27     19.7±0.94ms        ? ?/sec    1.00     15.5±0.77ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.00     11.6±0.24ms        ? ?/sec    1.02     11.8±0.75ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.00      9.8±0.13ms        ? ?/sec    1.01      9.9±0.31ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.6±0.27ms        ? ?/sec    1.01     11.7±0.45ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.8±0.15ms        ? ?/sec    1.01      9.9±0.31ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	790.2s
Peak memory	2.1 GiB
Avg memory	463.9 MiB
CPU user	2138.8s
CPU sys	106.2s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	785.2s
Peak memory	2.1 GiB
Avg memory	470.4 MiB
CPU user	1976.8s
CPU sys	116.1s
Peak spill	0 B

File an issue against this benchmark runner

alamb · 2026-07-16T19:41:29Z

🤔

These two seem to reproducibly get worse

distinct 10000000 rows asc [no TopK]                              1.00     31.8±0.48ms        ? ?/sec    1.65     52.4±2.91ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     31.8±0.30ms        ? ?/sec    1.70     53.9±2.74ms        ? ?/sec

Can you look into that @MassivePizza ?

MassivePizza · 2026-07-17T09:31:18Z

Only getting noise locally. I don't think my changes should affect "no TopK" regardless.

distinct 10000000 rows desc [no TopK]
                        time:   [29.979 ms 30.184 ms 30.388 ms]
                        change: [−0.4089% +0.4916% +1.3107%] (p = 0.27 > 0.05)
                        No change in performance detected.

distinct 10000000 rows asc [no TopK]
                        time:   [30.436 ms 30.733 ms 31.053 ms]
                        change: [+1.5647% +2.7108% +3.9667%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

distinct 10000000 rows desc [TopK]
                        time:   [4.5273 ms 4.5704 ms 4.6145 ms]
                        change: [−23.068% −22.347% −21.532%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

distinct 10000000 rows asc [TopK]
                        time:   [4.3722 ms 4.3967 ms 4.4224 ms]
                        change: [−25.664% −24.988% −24.372%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

avoiding from(Vec) into drop.

xudong963

LGTM

xudong963 · 2026-07-30T08:13:44Z

run benchmark topk_aggregate

xudong963 · 2026-07-30T08:14:06Z

🤔

These two seem to reproducibly get worse

distinct 10000000 rows asc [no TopK]                              1.00     31.8±0.48ms        ? ?/sec    1.65     52.4±2.91ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     31.8±0.30ms        ? ?/sec    1.70     53.9±2.74ms        ? ?/sec

Can you look into that @MassivePizza ?

Retriggered the benchmark, let's see if it can be reproduced

adriangbot · 2026-07-30T08:16:50Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5128314293-1298-wrfkh 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (5e07c8d) to e8a65f2 (merge-base) diff using: topk_aggregate
Results will be posted here when complete

File an issue against this benchmark runner

alamb · 2026-07-31T22:06:48Z

This is very nice @MassivePizza and @xudong963 and @jayzhan211 -- I think it is close.

adriangbot · 2026-07-31T22:48:01Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing concrete-topk-storage (5e07c8d) to e8a65f2 (merge-base) diff

Run configuration

run benchmark topk_aggregate

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     66.0±7.15ms        ? ?/sec    1.05     69.1±7.78ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     63.1±6.09ms        ? ?/sec    1.09     68.8±7.93ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.41      5.9±0.18ms        ? ?/sec    1.00      4.2±0.13ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.00     53.3±3.41ms        ? ?/sec    1.12     59.5±4.63ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.46      6.1±0.29ms        ? ?/sec    1.00      4.2±0.07ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     53.8±3.17ms        ? ?/sec    1.10     59.5±3.69ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.00     53.4±2.87ms        ? ?/sec    1.00     53.2±3.27ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.00     51.0±2.74ms        ? ?/sec    1.00     51.2±2.57ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   294.7±17.84ms        ? ?/sec    1.04   306.9±23.15ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   307.3±20.70ms        ? ?/sec    1.03   317.5±17.50ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.29     11.8±0.50ms        ? ?/sec    1.00      9.1±0.29ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.29     13.4±0.55ms        ? ?/sec    1.00     10.4±0.40ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.33     18.4±1.04ms        ? ?/sec    1.00     13.8±0.68ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.32     20.2±1.23ms        ? ?/sec    1.00     15.3±0.73ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.01     11.8±0.33ms        ? ?/sec    1.00     11.6±0.30ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.02     10.1±0.32ms        ? ?/sec    1.00      9.9±0.49ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.5±0.27ms        ? ?/sec    1.08     12.5±0.58ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.9±0.30ms        ? ?/sec    1.00      9.9±0.28ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1235.3s
Peak memory	2.2 GiB
Avg memory	291.6 MiB
CPU user	1986.3s
CPU sys	124.9s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1230.3s
Peak memory	2.1 GiB
Avg memory	296.5 MiB
CPU user	1932.5s
CPU sys	136.4s
Peak spill	0 B

File an issue against this benchmark runner

MassivePizza · 2026-08-03T12:16:16Z

This PR feels pretty solid on its own now.

~~I'll be testing a StringHashTable that doesn't clone any Strings in a separate PR.~~ Right, already tried that and it doesn't work because batches are temporary, silly me.

kumarUjjawal · 2026-08-03T14:38:01Z

run benchmark topk_aggregate

adriangbot · 2026-08-03T14:40:59Z

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5167784427-1352-jst7g 6.12.85+ #1 SMP Wed Jun 17 20:31:55 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing concrete-topk-storage (e5861d2) to 62650ef (merge-base) diff

Run configuration

run benchmark topk_aggregate

Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-03T15:20:06Z

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing concrete-topk-storage (e5861d2) to 62650ef (merge-base) diff

Run configuration

run benchmark topk_aggregate

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                             HEAD                                   concrete-topk-storage
-----                                                             ----                                   ---------------------
aggregate 10000000 time-series rows                               1.00     63.1±5.64ms        ? ?/sec    1.05     66.2±6.50ms        ? ?/sec
aggregate 10000000 worst-case rows                                1.00     60.6±5.40ms        ? ?/sec    1.06     64.4±7.67ms        ? ?/sec
distinct 10000000 rows asc [TopK]                                 1.38      5.9±0.10ms        ? ?/sec    1.00      4.3±0.11ms        ? ?/sec
distinct 10000000 rows asc [no TopK]                              1.02     57.5±3.90ms        ? ?/sec    1.00     56.2±3.87ms        ? ?/sec
distinct 10000000 rows desc [TopK]                                1.44      6.0±0.14ms        ? ?/sec    1.00      4.2±0.06ms        ? ?/sec
distinct 10000000 rows desc [no TopK]                             1.00     52.8±3.07ms        ? ?/sec    1.10     58.0±4.22ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8View]             1.00     52.7±2.69ms        ? ?/sec    1.04     54.6±3.03ms        ? ?/sec
string aggregate 10000000 time-series rows [Utf8]                 1.01     51.3±2.59ms        ? ?/sec    1.00     50.9±2.92ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8View]              1.00   312.7±18.80ms        ? ?/sec    1.06   331.3±19.82ms        ? ?/sec
string aggregate 10000000 worst-case rows [Utf8]                  1.00   324.3±22.88ms        ? ?/sec    1.02   331.8±19.08ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows                      1.28     12.2±0.49ms        ? ?/sec    1.00      9.5±0.31ms        ? ?/sec
top k=10 aggregate 10000000 time-series rows [Utf8View]           1.30     13.9±0.55ms        ? ?/sec    1.00     10.7±0.48ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows                       1.31     19.0±1.33ms        ? ?/sec    1.00     14.4±0.74ms        ? ?/sec
top k=10 aggregate 10000000 worst-case rows [Utf8View]            1.30     20.7±1.19ms        ? ?/sec    1.00     15.9±0.94ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8View]    1.01     11.8±0.26ms        ? ?/sec    1.00     11.7±0.30ms        ? ?/sec
top k=10 string aggregate 10000000 time-series rows [Utf8]        1.01      9.9±0.28ms        ? ?/sec    1.00      9.8±0.23ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8View]     1.00     11.9±0.44ms        ? ?/sec    1.04     12.3±0.28ms        ? ?/sec
top k=10 string aggregate 10000000 worst-case rows [Utf8]         1.00      9.8±0.18ms        ? ?/sec    1.01     10.0±0.25ms        ? ?/sec

Resource Usage

topk_aggregate — base (merge-base)

Metric	Value
Wall time	1170.3s
Peak memory	2.2 GiB
Avg memory	308.3 MiB
CPU user	1982.6s
CPU sys	118.5s
Peak spill	0 B

topk_aggregate — branch

Metric	Value
Wall time	1165.3s
Peak memory	2.1 GiB
Avg memory	308.0 MiB
CPU user	1974.9s
CPU sys	133.0s
Peak spill	0 B

File an issue against this benchmark runner

kumarUjjawal · 2026-08-04T04:31:15Z

@MassivePizza can you resolve the clippy issue

MassivePizza · 2026-08-04T12:51:02Z

I just realized that the benches are extra flaky due to heavy multithreading, which seems to affect string aggregate the most. My M4 Mac throttles quite quickly on those, but I'm certain the StringArrayType change is not a regression otherwise.

alamb · 2026-08-11T20:51:25Z

Is this one ready to merge?

MassivePizza · 2026-08-12T12:17:07Z

Sure thing. The mentioned PR is a follow-up.

xudong963 · 2026-08-12T16:04:47Z

THanks for your review @alamb @jayzhan211 and thanks for your first contribution!

This restores behavior from before apache#23609.

apache#24105) ## Which issue does this PR close?  N/A. Continuation of apache#23609. ## Rationale for this change  Improve perf of TopK. ## What changes are included in this PR?  Eliminate overhead of casting with `dyn Any` for every String in `topk::StringHeap`. Rearrange `HashTableItem<ID>` `Option`-ness to avoid unnecessary unwraps on lookups in `topk::TopKHashTable`. Rewrite inserts to take a borrowable type, delaying clones and allowing reuse of storage slots (mostly Strings). ## Are these changes tested?  Should be covered by existing tests. Making `ID` an `Option<ID>` inside `HashTableItem` should be fine, since the internal `HashTable` keeps track of valid items (although this could be hard to verify properly). It runs with the assumption that "if everything needed an unwrap, nothing needs an unwrap". ## Are there any user-facing changes?  No.  --------- Co-authored-by: kosiew <kosiew@gmail.com>

MassivePizza added 4 commits July 15, 2026 16:58

store concrete type in TopK structures

2a56ad5

store concrete string type in TopK hash-table

14a5c02

swap in-place in TopK heap

293a0e8

fix error from cherry-pick

3b28922

github-actions Bot added the physical-plan Changes to the physical-plan crate label Jul 15, 2026

jayzhan211 approved these changes Jul 16, 2026

View reviewed changes

MassivePizza added 2 commits July 16, 2026 17:40

fmt and clippy

6214dfa

Merge branch 'main' into concrete-topk-storage

8a7b678

alamb added the performance Make DataFusion faster label Jul 16, 2026

MassivePizza added 6 commits July 17, 2026 12:43

Merge branch 'main' into concrete-topk-storage

f44bbd9

Merge branch 'main' into concrete-topk-storage

4fc87b5

Merge branch 'main' into concrete-topk-storage

0069af0

remove redundant data_type in PrimitiveHashTable

a82f5fa

Undo StringArrayType convenience check,

dfbf3f9

avoiding from(Vec) into drop.

simplify set_batch in PrimitiveHashTable

5e07c8d

MassivePizza mentioned this pull request Jul 27, 2026

[X-3345] perf: Concrete Top-K storage massive-com/arrow-datafusion#69

Merged

xudong963 approved these changes Jul 30, 2026

View reviewed changes

MassivePizza added 3 commits August 3, 2026 13:47

replace StringArrayType enum with arrow trait

55a2e33

Merge remote-tracking branch 'upstream/main' into concrete-topk-storage

22747b1

use some_value helper

e5861d2

github-actions Bot removed the auto detected api change Auto detected API change label Aug 3, 2026

MassivePizza added 3 commits August 4, 2026 14:55

fix clippy

c5131bd

Merge branch 'main' into concrete-topk-storage

8e96042

share equality closures

71ec3cc

MassivePizza changed the title ~~perf: Concrete TopK array storage~~ perf: Concretely typed TopK ArrowHashTable storage Aug 5, 2026

MassivePizza mentioned this pull request Aug 5, 2026

perf: Concretely typed TopK StringHeap, and StringHashTable slot reuse #24105

Merged

Merge branch 'main' into concrete-topk-storage

b3f9734

xudong963 added this pull request to the merge queue Aug 12, 2026

Merged via the queue into apache:main with commit c81e707 Aug 12, 2026
40 checks passed

xudong963 deleted the concrete-topk-storage branch August 12, 2026 16:16

MassivePizza added a commit to massive-com/arrow-datafusion that referenced this pull request Aug 21, 2026

Store exact type in TopKHeap and TopKHashTable.

cb02950

This restores behavior from before apache#23609.

xudong963 mentioned this pull request Aug 23, 2026

Upgrade DataFusion fork to 55.0.0 and preserve fork patches massive-com/arrow-datafusion#73

Merged

MassivePizza added a commit to massive-com/arrow-datafusion that referenced this pull request Aug 27, 2026

Store exact type in TopKHeap and TopKHashTable.

06e34f7

This restores behavior from before apache#23609.

nuno-faria added the v56.0.0 label Oct 3, 2026

Conversation

MassivePizza commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

codecov-commenter commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

adriangbot commented Jul 16, 2026

Uh oh!

alamb commented Jul 16, 2026

Uh oh!

MassivePizza commented Jul 17, 2026

Uh oh!

xudong963 left a comment

Choose a reason for hiding this comment

Uh oh!

xudong963 commented Jul 30, 2026

Uh oh!

xudong963 commented Jul 30, 2026

Uh oh!

adriangbot commented Jul 30, 2026

Uh oh!

alamb commented Jul 31, 2026

Uh oh!

adriangbot commented Jul 31, 2026

Uh oh!

MassivePizza commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kumarUjjawal commented Aug 3, 2026

Uh oh!

adriangbot commented Aug 3, 2026

Uh oh!

adriangbot commented Aug 3, 2026

Uh oh!

kumarUjjawal commented Aug 4, 2026

Uh oh!

MassivePizza commented Aug 4, 2026

Uh oh!

alamb commented Aug 11, 2026

Uh oh!

MassivePizza commented Aug 12, 2026

Uh oh!

xudong963 commented Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

MassivePizza commented Jul 15, 2026 •

edited

Loading

codecov-commenter commented Jul 16, 2026 •

edited

Loading

MassivePizza commented Aug 3, 2026 •

edited

Loading