Skip to content

perf: Optimize NULL handling in some datetime functions#21477

Merged
mbutrovich merged 2 commits intoapache:mainfrom
neilconway:neilc/perf-datetime-nulls
Apr 10, 2026
Merged

perf: Optimize NULL handling in some datetime functions#21477
mbutrovich merged 2 commits intoapache:mainfrom
neilconway:neilc/perf-datetime-nulls

Conversation

@neilconway
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

to_time, to_local_time, and make_date build NULLs arrays incrementally. It is faster and more idiomatic to build them in bulk via NullBuffer instead.

This doesn't make a huge performance difference in practice, but it shouldn't be slower, and the result is more concise and idiomatic.

Benchmarks (ARM64):

  - make_date_col_col_col_8192: 97.0µs -> 97.3µs, +0.3%
  - make_date_scalar_col_col_8192: 97.3µs -> 97.8µs, +0.5%
  - make_date_scalar_scalar_col_8192: 98.1µs -> 98.7µs, +0.6%
  - make_date_scalar_scalar_scalar: 162.0ns -> 162.1ns, +0.1%
  - to_time_no_nulls_100k: 17.1ms -> 17.7ms, +3.5%
  - to_time_10pct_nulls_100k: 16.0ms -> 16.2ms, +1.3%
  - to_local_time_no_nulls_100k: 8.0ms -> 7.8ms, -2.5%
  - to_local_time_10pct_nulls_100k: 7.6ms -> 7.0ms, -7.9%

What changes are included in this PR?

  • Add benchmarks for to_time and to_local_time
  • Use 8192 batch size in make_date benchmark, for consistency with default DF batch size
  • Implement NULL handling optimization/cleanup

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

Copy link
Copy Markdown
Contributor

@mbutrovich mbutrovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @neilconway!


let mut builder: PrimitiveBuilder<Time32SecondType> =
PrimitiveArray::builder(len);
let nulls = NullBuffer::union(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another for apache/arrow-rs#9692 eventually.

@mbutrovich mbutrovich added this pull request to the merge queue Apr 10, 2026
Merged via the queue into apache:main with commit 1929d71 Apr 10, 2026
34 checks passed
@neilconway neilconway deleted the neilc/perf-datetime-nulls branch April 10, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize NULL handling in some datetime functions

2 participants