[Parquet] Vectorise Pack Bits within BitWriter by sdf-jkl · Pull Request #10432 · apache/arrow-rs

sdf-jkl · 2026-07-24T22:05:04Z

Which issue does this PR close?

Closes Vectorised Pack Bits within BitWriter #2257.

Rationale for this change

Perf 🚀

What changes are included in this PR?

Renamed the FromBitpacked to BitPacking to store the Packing APIs
Added vectorised pack_batch matching unpack_batch kernels
Added unit tests
Added bench arms

Are these changes tested?

Yes!

Are there any user-facing changes?

No

sdf-jkl · 2026-07-25T00:35:33Z

run benchmark arrow_writer

adriangbot · 2026-07-25T00:38:31Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5075864703-1263-hj9x2 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing put_batch (f462d31) to f7dfcd2 (merge-base) diff
BENCH_NAME=arrow_writer
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
BENCH_FILTER=
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-07-25T02:18:10Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   put_batch
-----                                              ----                                   ---------
bool/bloom_filter                                  1.01     13.0±0.06ms    19.3 MB/sec    1.00     12.8±0.10ms    19.5 MB/sec
bool/cdc                                           1.01     15.7±0.07ms    15.9 MB/sec    1.00     15.5±0.11ms    16.1 MB/sec
bool/default                                       1.01     10.8±0.06ms    23.1 MB/sec    1.00     10.7±0.10ms    23.4 MB/sec
bool/parquet_2                                     1.04     14.6±0.07ms    17.1 MB/sec    1.00     14.0±0.09ms    17.9 MB/sec
bool/zstd                                          1.02     11.3±0.04ms    22.1 MB/sec    1.00     11.2±0.10ms    22.4 MB/sec
bool/zstd_parquet_2                                1.04     15.0±0.05ms    16.7 MB/sec    1.00     14.3±0.10ms    17.4 MB/sec
bool_non_null/bloom_filter                         1.08      7.0±0.03ms    18.0 MB/sec    1.00      6.5±0.03ms    19.4 MB/sec
bool_non_null/cdc                                  1.09      6.8±0.05ms    18.4 MB/sec    1.00      6.2±0.03ms    20.1 MB/sec
bool_non_null/default                              1.13      4.2±0.03ms    29.9 MB/sec    1.00      3.7±0.02ms    34.0 MB/sec
bool_non_null/parquet_2                            1.11      8.9±0.03ms    14.0 MB/sec    1.00      8.0±0.03ms    15.6 MB/sec
bool_non_null/zstd                                 1.13      4.5±0.04ms    27.6 MB/sec    1.00      4.0±0.03ms    31.1 MB/sec
bool_non_null/zstd_parquet_2                       1.11      9.3±0.04ms    13.4 MB/sec    1.00      8.4±0.03ms    14.9 MB/sec
bool_ree/bloom_filter                              1.00     30.4±0.08ms     7.0 MB/sec    1.00     30.3±0.15ms     7.0 MB/sec
bool_ree/cdc                                       1.00     31.7±0.09ms     6.7 MB/sec    1.00     31.6±0.12ms     6.7 MB/sec
bool_ree/default                                   1.00     27.7±0.08ms     7.7 MB/sec    1.00     27.6±0.14ms     7.7 MB/sec
bool_ree/parquet_2                                 1.00     29.3±0.10ms     7.3 MB/sec    1.01     29.5±0.13ms     7.2 MB/sec
bool_ree/zstd                                      1.00     28.3±0.08ms     7.5 MB/sec    1.00     28.2±0.14ms     7.6 MB/sec
bool_ree/zstd_parquet_2                            1.00     29.6±0.09ms     7.2 MB/sec    1.01     29.8±0.15ms     7.1 MB/sec
decimal/bloom_filter                               1.00    146.5±0.60ms   409.6 MB/sec    1.01    148.0±0.78ms   405.5 MB/sec
decimal/cdc                                        1.00    131.6±0.43ms   456.1 MB/sec    1.01    132.3±0.21ms   453.4 MB/sec
decimal/default                                    1.00    107.0±0.48ms   560.5 MB/sec    1.00    107.3±0.35ms   559.3 MB/sec
decimal/parquet_2                                  1.01    272.3±0.91ms   220.4 MB/sec    1.00    269.0±0.68ms   223.0 MB/sec
decimal/zstd                                       1.00    118.2±0.40ms   507.7 MB/sec    1.00    118.1±0.40ms   508.1 MB/sec
decimal/zstd_parquet_2                             1.01    292.1±0.71ms   205.4 MB/sec    1.00    289.2±0.60ms   207.4 MB/sec
fixed_size_binary_ree/bloom_filter                 1.00     69.3±0.36ms    14.5 MB/sec    1.00     69.6±0.27ms    14.5 MB/sec
fixed_size_binary_ree/cdc                          1.00     72.8±0.15ms    13.8 MB/sec    1.00     73.0±0.18ms    13.8 MB/sec
fixed_size_binary_ree/default                      1.00     62.0±0.20ms    16.2 MB/sec    1.01     62.6±0.24ms    16.1 MB/sec
fixed_size_binary_ree/parquet_2                    1.00     79.2±0.51ms    12.7 MB/sec    1.02     80.8±0.31ms    12.5 MB/sec
fixed_size_binary_ree/zstd                         1.00     67.7±0.27ms    14.9 MB/sec    1.01     68.2±0.23ms    14.8 MB/sec
fixed_size_binary_ree/zstd_parquet_2               1.00     79.9±0.56ms    12.6 MB/sec    1.02     81.8±0.39ms    12.3 MB/sec
float_with_nans/bloom_filter                       1.00     92.2±0.22ms   151.8 MB/sec    1.01     92.7±0.26ms   151.0 MB/sec
float_with_nans/cdc                                1.00     81.0±0.21ms   172.8 MB/sec    1.01     82.0±0.17ms   170.8 MB/sec
float_with_nans/default                            1.00     73.8±0.20ms   189.7 MB/sec    1.01     74.5±0.13ms   188.0 MB/sec
float_with_nans/parquet_2                          1.00     94.3±0.22ms   148.5 MB/sec    1.02     96.6±0.25ms   144.9 MB/sec
float_with_nans/zstd                               1.00    111.9±0.16ms   125.2 MB/sec    1.01    112.6±0.20ms   124.3 MB/sec
float_with_nans/zstd_parquet_2                     1.00    131.4±0.34ms   106.5 MB/sec    1.02    133.8±0.27ms   104.6 MB/sec
fsb/bloom_filter                                   1.00      9.7±0.11ms  1654.5 MB/sec    1.00      9.7±0.11ms  1654.3 MB/sec
fsb/cdc                                            1.01     12.1±0.07ms  1328.5 MB/sec    1.00     12.0±0.06ms  1344.0 MB/sec
fsb/default                                        1.00      8.5±0.06ms  1896.2 MB/sec    1.00      8.5±0.05ms  1900.0 MB/sec
fsb/parquet_2                                      1.00     18.4±0.11ms   875.4 MB/sec    1.00     18.4±0.12ms   874.9 MB/sec
fsb/zstd                                           1.00     10.9±0.07ms  1478.0 MB/sec    1.02     11.2±0.10ms  1444.5 MB/sec
fsb/zstd_parquet_2                                 1.00     19.1±0.11ms   845.5 MB/sec    1.02     19.4±0.15ms   830.2 MB/sec
int32_ree/bloom_filter                             1.00     40.8±0.25ms    10.0 MB/sec    1.04     42.5±0.36ms     9.6 MB/sec
int32_ree/cdc                                      1.00     40.3±0.08ms    10.1 MB/sec    1.03     41.7±0.13ms     9.8 MB/sec
int32_ree/default                                  1.00     36.4±0.09ms    11.2 MB/sec    1.04     37.9±0.13ms    10.7 MB/sec
int32_ree/parquet_2                                1.00     36.4±0.09ms    11.2 MB/sec    1.04     37.9±0.13ms    10.7 MB/sec
int32_ree/zstd                                     1.00     37.0±0.07ms    11.0 MB/sec    1.04     38.5±0.14ms    10.6 MB/sec
int32_ree/zstd_parquet_2                           1.00     36.9±0.10ms    11.0 MB/sec    1.04     38.4±0.13ms    10.6 MB/sec
int32_ree_95pct_null/bloom_filter                  1.00     22.0±0.09ms    18.5 MB/sec    1.01     22.1±0.07ms    18.4 MB/sec
int32_ree_95pct_null/cdc                           1.01     24.2±0.05ms    16.8 MB/sec    1.00     23.9±0.05ms    17.0 MB/sec
int32_ree_95pct_null/default                       1.00     21.6±0.04ms    18.9 MB/sec    1.00     21.7±0.03ms    18.8 MB/sec
int32_ree_95pct_null/parquet_2                     1.00     21.6±0.03ms    18.8 MB/sec    1.01     21.7±0.06ms    18.7 MB/sec
int32_ree_95pct_null/zstd                          1.00     21.9±0.04ms    18.6 MB/sec    1.00     21.9±0.04ms    18.6 MB/sec
int32_ree_95pct_null/zstd_parquet_2                1.00     21.8±0.03ms    18.7 MB/sec    1.00     21.8±0.03ms    18.6 MB/sec
large_string_non_null/bloom_filter                 1.00     71.7±0.13ms     3.5 GB/sec    1.02     72.9±0.11ms     3.4 GB/sec
large_string_non_null/cdc                          1.00    243.2±1.17ms  1052.5 MB/sec    1.00    244.2±1.02ms  1048.2 MB/sec
large_string_non_null/default                      1.00     53.8±0.11ms     4.6 GB/sec    1.02     55.0±0.11ms     4.5 GB/sec
large_string_non_null/parquet_2                    1.00     53.7±0.10ms     4.7 GB/sec    1.03     55.2±0.12ms     4.5 GB/sec
large_string_non_null/zstd                         1.00     53.8±0.10ms     4.6 GB/sec    1.02     55.1±0.10ms     4.5 GB/sec
large_string_non_null/zstd_parquet_2               1.00     53.9±0.11ms     4.6 GB/sec    1.02     55.2±0.11ms     4.5 GB/sec
list_nested/bloom_filter                           1.00    148.3±0.21ms   194.5 MB/sec    1.00    148.6±0.23ms   194.2 MB/sec
list_nested/cdc                                    1.01    165.4±0.17ms   174.4 MB/sec    1.00    163.8±0.22ms   176.2 MB/sec
list_nested/default                                1.01    133.6±0.22ms   215.9 MB/sec    1.00    132.8±0.20ms   217.2 MB/sec
list_nested/parquet_2                              1.02    146.8±0.23ms   196.5 MB/sec    1.00    143.9±0.19ms   200.5 MB/sec
list_nested/zstd                                   1.00    142.8±0.18ms   202.1 MB/sec    1.00    142.3±0.42ms   202.8 MB/sec
list_nested/zstd_parquet_2                         1.02    159.6±0.19ms   180.8 MB/sec    1.00    156.7±0.23ms   184.2 MB/sec
list_primitive/bloom_filter                        1.00    282.0±1.19ms  1933.8 MB/sec    1.00    281.5±1.89ms  1937.2 MB/sec
list_primitive/cdc                                 1.01    315.4±0.96ms  1729.3 MB/sec    1.00    312.6±2.01ms  1744.7 MB/sec
list_primitive/default                             1.02    205.8±0.94ms     2.6 GB/sec    1.00    200.9±1.63ms     2.7 GB/sec
list_primitive/parquet_2                           1.03    227.9±0.48ms     2.3 GB/sec    1.00    221.6±0.43ms     2.4 GB/sec
list_primitive/zstd                                1.01    461.1±0.74ms  1182.7 MB/sec    1.00    457.5±2.78ms  1192.1 MB/sec
list_primitive/zstd_parquet_2                      1.01    450.8±1.04ms  1209.8 MB/sec    1.00    444.9±0.37ms  1225.7 MB/sec
list_primitive_non_null/bloom_filter               1.06    372.0±5.04ms  1463.1 MB/sec    1.00    350.7±4.58ms  1551.8 MB/sec
list_primitive_non_null/cdc                        1.00    395.4±6.87ms  1376.5 MB/sec    1.00    395.0±7.70ms  1378.0 MB/sec
list_primitive_non_null/default                    1.11    241.7±2.82ms     2.2 GB/sec    1.00    217.8±4.92ms     2.4 GB/sec
list_primitive_non_null/parquet_2                  1.11    272.8±0.65ms  1994.8 MB/sec    1.00    246.3±0.56ms     2.2 GB/sec
list_primitive_non_null/zstd                       1.03    665.3±2.64ms   818.1 MB/sec    1.00    647.5±4.98ms   840.5 MB/sec
list_primitive_non_null/zstd_parquet_2             1.03    649.6±8.85ms   837.7 MB/sec    1.00    631.0±3.47ms   862.4 MB/sec
list_primitive_sparse_99pct_null/bloom_filter      1.00     12.8±0.09ms     2.9 GB/sec    1.01     12.8±0.12ms     2.8 GB/sec
list_primitive_sparse_99pct_null/cdc               1.01     24.6±0.12ms  1520.0 MB/sec    1.00     24.5±0.10ms  1527.7 MB/sec
list_primitive_sparse_99pct_null/default           1.00     12.4±0.06ms     2.9 GB/sec    1.01     12.5±0.07ms     2.9 GB/sec
list_primitive_sparse_99pct_null/parquet_2         1.00     12.4±0.05ms     2.9 GB/sec    1.00     12.5±0.07ms     2.9 GB/sec
list_primitive_sparse_99pct_null/zstd              1.00     14.3±0.07ms     2.5 GB/sec    1.00     14.4±0.09ms     2.5 GB/sec
list_primitive_sparse_99pct_null/zstd_parquet_2    1.00     12.6±0.09ms     2.9 GB/sec    1.00     12.6±0.11ms     2.9 GB/sec
list_struct_with_list/bloom_filter                 1.00    283.2±0.46ms   161.0 MB/sec    1.01    284.8±0.52ms   160.1 MB/sec
list_struct_with_list/cdc                          1.00    324.2±0.46ms   140.6 MB/sec    1.00    323.3±0.52ms   141.0 MB/sec
list_struct_with_list/default                      1.00    257.9±0.39ms   176.8 MB/sec    1.00    258.5±0.38ms   176.4 MB/sec
list_struct_with_list/parquet_2                    1.01    275.5±0.37ms   165.5 MB/sec    1.00    273.3±0.39ms   166.9 MB/sec
list_struct_with_list/zstd                         1.00    280.4±0.88ms   162.6 MB/sec    1.00    280.7±0.45ms   162.4 MB/sec
list_struct_with_list/zstd_parquet_2               1.01    298.5±0.42ms   152.8 MB/sec    1.00    296.5±0.37ms   153.8 MB/sec
primitive/bloom_filter                             1.00    148.4±0.55ms   302.5 MB/sec    1.02    150.8±0.67ms   297.6 MB/sec
primitive/cdc                                      1.00    158.1±0.70ms   283.9 MB/sec    1.02    160.5±0.66ms   279.7 MB/sec
primitive/default                                  1.00    116.3±0.44ms   385.8 MB/sec    1.03    119.5±0.59ms   375.4 MB/sec
primitive/parquet_2                                1.01    131.5±0.50ms   341.2 MB/sec    1.00    130.5±0.66ms   343.9 MB/sec
primitive/zstd                                     1.00    146.8±0.66ms   305.7 MB/sec    1.02    149.4±0.70ms   300.4 MB/sec
primitive/zstd_parquet_2                           1.01    165.3±2.57ms   271.4 MB/sec    1.00    163.6±0.63ms   274.4 MB/sec
primitive_all_null/bloom_filter                    1.00    880.6±2.84µs    49.8 GB/sec    1.01    892.2±6.10µs    49.1 GB/sec
primitive_all_null/cdc                             1.01     19.1±0.36ms     2.3 GB/sec    1.00     18.9±0.32ms     2.3 GB/sec
primitive_all_null/default                         1.00    267.8±1.32µs   163.7 GB/sec    1.01    270.6±1.11µs   161.9 GB/sec
primitive_all_null/parquet_2                       1.00    267.9±1.25µs   163.6 GB/sec    1.00    269.2±1.40µs   162.8 GB/sec
primitive_all_null/zstd                            1.00    418.7±1.23µs   104.7 GB/sec    1.00    417.0±1.13µs   105.1 GB/sec
primitive_all_null/zstd_parquet_2                  1.00    366.1±1.32µs   119.7 GB/sec    1.00    367.5±1.15µs   119.3 GB/sec
primitive_non_null/bloom_filter                    1.00    106.6±0.34ms   412.9 MB/sec    1.00    107.0±0.41ms   411.2 MB/sec
primitive_non_null/cdc                             1.00     89.0±0.28ms   494.6 MB/sec    1.03     91.5±0.33ms   480.7 MB/sec
primitive_non_null/default                         1.00     67.0±0.22ms   656.7 MB/sec    1.01     67.9±0.25ms   648.3 MB/sec
primitive_non_null/parquet_2                       1.05     88.6±0.20ms   496.6 MB/sec    1.00     84.0±0.26ms   523.6 MB/sec
primitive_non_null/zstd                            1.00     97.9±0.19ms   449.2 MB/sec    1.01     98.6±0.24ms   446.3 MB/sec
primitive_non_null/zstd_parquet_2                  1.04    121.7±0.22ms   361.7 MB/sec    1.00    117.2±0.30ms   375.3 MB/sec
primitive_sparse_99pct_null/bloom_filter           1.00     12.2±0.18ms     3.6 GB/sec    1.02     12.4±0.16ms     3.5 GB/sec
primitive_sparse_99pct_null/cdc                    1.02     30.5±0.34ms  1471.3 MB/sec    1.00     30.0±0.35ms  1497.2 MB/sec
primitive_sparse_99pct_null/default                1.00     10.7±0.06ms     4.1 GB/sec    1.01     10.8±0.07ms     4.1 GB/sec
primitive_sparse_99pct_null/parquet_2              1.00     10.7±0.05ms     4.1 GB/sec    1.02     10.9±0.10ms     4.0 GB/sec
primitive_sparse_99pct_null/zstd                   1.00     14.1±0.13ms     3.1 GB/sec    1.01     14.3±0.10ms     3.1 GB/sec
primitive_sparse_99pct_null/zstd_parquet_2         1.00     12.6±0.08ms     3.5 GB/sec    1.01     12.7±0.10ms     3.4 GB/sec
short_string_non_null/bloom_filter                 1.02     27.2±0.05ms   441.1 MB/sec    1.00     26.6±0.13ms   451.9 MB/sec
short_string_non_null/cdc                          1.00     19.5±0.04ms   615.8 MB/sec    1.01     19.6±0.05ms   610.7 MB/sec
short_string_non_null/default                      1.00     15.4±0.04ms   778.5 MB/sec    1.01     15.6±0.07ms   768.4 MB/sec
short_string_non_null/parquet_2                    1.07     26.7±0.12ms   448.8 MB/sec    1.00     24.9±0.09ms   482.4 MB/sec
short_string_non_null/zstd                         1.00     35.6±0.11ms   337.1 MB/sec    1.00     35.8±0.10ms   335.6 MB/sec
short_string_non_null/zstd_parquet_2               1.07     29.6±0.11ms   405.2 MB/sec    1.00     27.8±0.06ms   432.2 MB/sec
string/bloom_filter                                1.03   230.3±24.68ms     2.2 GB/sec    1.00   223.5±23.54ms     2.3 GB/sec
string/cdc                                         1.00    202.3±1.01ms     2.5 GB/sec    1.00    202.8±0.96ms     2.5 GB/sec
string/default                                     1.13   136.8±25.68ms     3.7 GB/sec    1.00   120.9±20.19ms     4.2 GB/sec
string/parquet_2                                   1.00    101.9±6.29ms     5.0 GB/sec    1.11    113.1±0.32ms     4.5 GB/sec
string/zstd                                        1.00    430.1±2.55ms  1218.9 MB/sec    1.07   461.9±12.13ms  1134.9 MB/sec
string/zstd_parquet_2                              1.00    384.4±0.84ms  1363.7 MB/sec    1.00    383.1±3.30ms  1368.5 MB/sec
string_and_binary_view/bloom_filter                1.00     63.5±0.20ms   507.9 MB/sec    1.04     65.8±0.24ms   490.4 MB/sec
string_and_binary_view/cdc                         1.00     58.7±0.32ms   549.2 MB/sec    1.02     60.1±0.22ms   536.6 MB/sec
string_and_binary_view/default                     1.00     48.0±0.13ms   672.3 MB/sec    1.03     49.3±0.17ms   653.9 MB/sec
string_and_binary_view/parquet_2                   1.01     59.8±0.17ms   539.2 MB/sec    1.00     59.3±0.16ms   544.1 MB/sec
string_and_binary_view/zstd                        1.00     85.6±0.30ms   376.8 MB/sec    1.01     86.3±0.19ms   373.6 MB/sec
string_and_binary_view/zstd_parquet_2              1.01     73.9±0.29ms   436.6 MB/sec    1.00     73.1±0.18ms   441.0 MB/sec
string_dictionary/bloom_filter                     1.00     84.1±0.46ms     3.1 GB/sec    1.02     86.1±0.73ms     3.0 GB/sec
string_dictionary/cdc                              1.00     48.2±0.68ms     5.4 GB/sec    1.03     49.5±0.48ms     5.2 GB/sec
string_dictionary/default                          1.00     43.2±0.30ms     6.0 GB/sec    1.03     44.5±0.39ms     5.8 GB/sec
string_dictionary/parquet_2                        1.02     48.7±0.11ms     5.3 GB/sec    1.00     47.8±0.12ms     5.4 GB/sec
string_dictionary/zstd                             1.00    209.4±1.16ms  1261.5 MB/sec    1.00    209.5±0.59ms  1260.7 MB/sec
string_dictionary/zstd_parquet_2                   1.00    192.0±0.19ms  1375.9 MB/sec    1.00    192.3±0.27ms  1373.4 MB/sec
string_non_null/bloom_filter                       1.01   233.0±10.62ms     2.2 GB/sec    1.00   230.7±10.62ms     2.2 GB/sec
string_non_null/cdc                                1.01    238.6±1.30ms     2.1 GB/sec    1.00    236.8±1.20ms     2.2 GB/sec
string_non_null/default                            1.00   116.7±12.55ms     4.4 GB/sec    1.01    117.3±9.64ms     4.4 GB/sec
string_non_null/parquet_2                          1.18    122.3±2.32ms     4.2 GB/sec    1.00    104.0±1.33ms     4.9 GB/sec
string_non_null/zstd                               1.04    557.1±3.70ms   940.5 MB/sec    1.00    534.8±1.86ms   979.8 MB/sec
string_non_null/zstd_parquet_2                     1.04    505.6±7.04ms  1036.4 MB/sec    1.00    485.4±2.16ms  1079.5 MB/sec
string_ree/bloom_filter                            1.00    147.7±0.74ms   111.1 MB/sec    1.02    150.8±0.57ms   108.8 MB/sec
string_ree/cdc                                     1.00    218.8±0.85ms    75.0 MB/sec    1.01    221.2±0.83ms    74.2 MB/sec
string_ree/default                                 1.00    120.7±0.50ms   135.9 MB/sec    1.02    123.0±0.66ms   133.4 MB/sec
string_ree/parquet_2                               1.00    173.3±0.47ms    94.7 MB/sec    1.01    175.1±0.34ms    93.7 MB/sec
string_ree/zstd                                    1.00    180.7±0.68ms    90.8 MB/sec    1.01    181.9±0.50ms    90.2 MB/sec
string_ree/zstd_parquet_2                          1.00    190.0±0.45ms    86.4 MB/sec    1.00    190.0±0.46ms    86.4 MB/sec
string_ree_95pct_null/bloom_filter                 1.01     26.7±0.08ms    52.7 MB/sec    1.00     26.5±0.14ms    53.1 MB/sec
string_ree_95pct_null/cdc                          1.01     32.9±0.14ms    42.8 MB/sec    1.00     32.7±0.13ms    43.0 MB/sec
string_ree_95pct_null/default                      1.00     25.0±0.05ms    56.2 MB/sec    1.00     24.9±0.09ms    56.4 MB/sec
string_ree_95pct_null/parquet_2                    1.00     25.0±0.05ms    56.2 MB/sec    1.00     24.9±0.08ms    56.4 MB/sec
string_ree_95pct_null/zstd                         1.00     25.9±0.08ms    54.4 MB/sec    1.00     25.7±0.09ms    54.6 MB/sec
string_ree_95pct_null/zstd_parquet_2               1.00     25.8±0.06ms    54.5 MB/sec    1.00     25.8±0.07ms    54.5 MB/sec
struct_all_null/bloom_filter                       1.00    369.9±1.92µs    42.6 GB/sec    1.00    370.2±2.04µs    42.5 GB/sec
struct_all_null/cdc                                1.00      7.8±0.07ms     2.0 GB/sec    1.00      7.8±0.14ms     2.0 GB/sec
struct_all_null/default                            1.01    116.5±0.52µs   135.2 GB/sec    1.00    115.6±0.57µs   136.2 GB/sec
struct_all_null/parquet_2                          1.00    115.5±0.58µs   136.3 GB/sec    1.01    116.1±0.55µs   135.6 GB/sec
struct_all_null/zstd                               1.01    180.9±0.53µs    87.0 GB/sec    1.00    179.4±0.53µs    87.8 GB/sec
struct_all_null/zstd_parquet_2                     1.00    157.0±0.49µs   100.3 GB/sec    1.01    158.4±0.68µs    99.4 GB/sec
struct_non_null/bloom_filter                       1.00     45.0±0.13ms   355.2 MB/sec    1.03     46.5±0.15ms   344.4 MB/sec
struct_non_null/cdc                                1.00     45.0±0.18ms   355.7 MB/sec    1.02     46.0±0.18ms   348.0 MB/sec
struct_non_null/default                            1.00     31.4±0.11ms   509.6 MB/sec    1.04     32.7±0.14ms   488.6 MB/sec
struct_non_null/parquet_2                          1.03     40.1±0.11ms   398.8 MB/sec    1.00     39.1±0.12ms   408.8 MB/sec
struct_non_null/zstd                               1.00     40.0±0.11ms   400.0 MB/sec    1.03     41.2±0.15ms   388.0 MB/sec
struct_non_null/zstd_parquet_2                     1.01     53.8±0.14ms   297.4 MB/sec    1.00     53.0±0.14ms   301.7 MB/sec
struct_sparse_99pct_null/bloom_filter              1.00      6.5±0.07ms     2.4 GB/sec    1.01      6.6±0.07ms     2.4 GB/sec
struct_sparse_99pct_null/cdc                       1.03     13.9±0.13ms  1156.9 MB/sec    1.00     13.5±0.14ms  1196.7 MB/sec
struct_sparse_99pct_null/default                   1.00      6.0±0.05ms     2.6 GB/sec    1.00      6.0±0.03ms     2.6 GB/sec
struct_sparse_99pct_null/parquet_2                 1.00      6.0±0.05ms     2.6 GB/sec    1.01      6.0±0.03ms     2.6 GB/sec
struct_sparse_99pct_null/zstd                      1.00      7.4±0.06ms     2.1 GB/sec    1.00      7.4±0.03ms     2.1 GB/sec
struct_sparse_99pct_null/zstd_parquet_2            1.00      6.7±0.05ms     2.3 GB/sec    1.01      6.8±0.04ms     2.3 GB/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	2960.7s
Peak memory	3.0 GiB
Avg memory	2.7 GiB
CPU user	2877.4s
CPU sys	77.9s
Peak spill	0 B

branch

Metric	Value
Wall time	2950.6s
Peak memory	2.9 GiB
Avg memory	2.7 GiB
CPU user	2881.3s
CPU sys	67.2s
Peak spill	0 B

File an issue against this benchmark runner

sdf-jkl · 2026-07-27T21:47:22Z

run benchmark arrow_writer

adriangbot · 2026-07-27T21:49:58Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5097175518-1278-2jpd7 6.12.85+ #1 SMP Mon May 11 08:17:35 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing put_batch (704d990) to 25569a3 (merge-base) diff
BENCH_NAME=arrow_writer
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_writer
BENCH_FILTER=
Results will be posted here when complete

File an issue against this benchmark runner

simdize bit_packing and move it under the same trait as unpack

aac9013

github-actions Bot added the parquet Changes to the parquet crate label Jul 24, 2026

fmt

f462d31

sdf-jkl added 2 commits July 27, 2026 17:46

restructure RLE to support put_batch

a276281

Merge branch 'main' of https://github.com/apache/arrow-rs into put_batch

704d990

fmt

978803c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Parquet] Vectorise Pack Bits within BitWriter - #10432

[Parquet] Vectorise Pack Bits within BitWriter#10432
sdf-jkl wants to merge 5 commits into
apache:mainfrom
sdf-jkl:put_batch

sdf-jkl commented Jul 24, 2026

Uh oh!

sdf-jkl commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

sdf-jkl commented Jul 27, 2026

Uh oh!

adriangbot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

sdf-jkl commented Jul 24, 2026

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

sdf-jkl commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

adriangbot commented Jul 25, 2026

Uh oh!

sdf-jkl commented Jul 27, 2026

Uh oh!

adriangbot commented Jul 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants