Skip to content

perf[buffer]: iteration for fallible operations with validity#8120

Open
joseph-isaacs wants to merge 12 commits into
developfrom
ji/fast-iter-valid
Open

perf[buffer]: iteration for fallible operations with validity#8120
joseph-isaacs wants to merge 12 commits into
developfrom
ji/fast-iter-valid

Conversation

@joseph-isaacs
Copy link
Copy Markdown
Contributor

@joseph-isaacs joseph-isaacs commented May 27, 2026

Currently use (and arrow) handle fallible operations with scalar (non-SIMD) code.

This PR add a trait and methods to have fast SIMD checked operations (includes cast) but verified else where that checked_add benefits

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 27, 2026

Merging this PR will degrade performance by 16.44%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

❌ 1 regressed benchmark
✅ 1265 untouched benchmarks
🆕 43 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 Simulation cast_u32_to_u8[65536] N/A 286.6 µs N/A
Simulation chunked_varbinview_opt_canonical_into[(1000, 10)] 187.8 µs 224.8 µs -16.44%
🆕 Simulation cast_i32_to_u32[65536] N/A 1 ms N/A
🆕 Simulation try_map_with_mask_narrow_u64_u32_lazy_validity[4096] N/A 32.2 µs N/A
🆕 Simulation try_map_with_mask_narrow_u64_u32_lazy_validity[65536] N/A 494.1 µs N/A
🆕 Simulation try_map_with_mask_narrow_u64_u32_value_only_filtered[4096] N/A 28.7 µs N/A
🆕 Simulation arrow_cast_widen_u16_u32[4096] N/A 36.5 µs N/A
🆕 Simulation map_with_mask_in_place_u32_zero_nulls[65536] N/A 153.4 µs N/A
🆕 Simulation map_with_mask_widen_u16_u32_zero_nulls[1048576] N/A 3.4 ms N/A
🆕 Simulation try_map_no_validity_narrow_u64_u32[65536] N/A 424.1 µs N/A
🆕 Simulation try_map_with_mask_widen_u16_u32_maskless[4096] N/A 12.7 µs N/A
🆕 Simulation map_no_validity_widen_u16_u32[1048576] N/A 3 ms N/A
🆕 Simulation bitpack_value_only[2097152] N/A 14.3 ms N/A
🆕 Simulation try_map_with_mask_narrow_u64_u32_ignoring_valid[4096] N/A 28.8 µs N/A
🆕 Simulation bitpack_value_only[4194304] N/A 28.7 ms N/A
🆕 Simulation try_map_with_mask_widen_u16_u32_maskless[1048576] N/A 3 ms N/A
🆕 Simulation try_map_with_mask_narrow_u64_u32_ignoring_valid[1048576] N/A 6.8 ms N/A
🆕 Simulation bitpack_value_only[4096] N/A 34 µs N/A
🆕 Simulation try_map_with_mask_narrow_u64_u32_lazy_validity[1048576] N/A 7.9 ms N/A
🆕 Simulation arrow_cast_narrow_u64_u32[65536] N/A 777.1 µs N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ji/fast-iter-valid (4299cf0) with develop (7bfe690)

Open in CodSpeed

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
f
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs changed the title faster iteration infra perf[buffer]: iteration for fallible operations with validity May 27, 2026
@joseph-isaacs joseph-isaacs marked this pull request as ready for review May 27, 2026 15:13
@joseph-isaacs
Copy link
Copy Markdown
Contributor Author

Open question is where to put this code?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant