-
-
Notifications
You must be signed in to change notification settings - Fork 14.4k
Optimize array chunks #151668
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Optimize array chunks #151668
Conversation
This comment has been minimized.
This comment has been minimized.
d853243 to
cc3e197
Compare
|
This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
|
r? @tgross35 |
This comment has been minimized.
This comment has been minimized.
cc3e197 to
a4f3496
Compare
| /// If `iter.next()` panics, all items already yielded by the iterator are | ||
| /// dropped. | ||
| /// Panic guard for incremental initialization of arrays from the back. | ||
| struct GuardBack<'a, T> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are you deleting documentation?
If this was AI-generated, please make sure you can explain every change in the diff.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've restored the missing documentation for iter_next_chunk and added detailed docs for the new backward-filling logic — sorry, it was a few custom config rules I added in my editor that trimmed them out. I'll make sure it doesn't happen again!
i implemented a new GuardBack specifically to handle the reverse-initialization logic safely. While the standard Guard fills an array from 0..N, GuardBack fills from N down to 0. If a panic occurs, GuardBack::drop correctly identifies and drops only the partial results at the end of the array (the len - initialized..len slice). This is required because we're pulling from the back of the iterator but want the final array to be in the original order.
I followed the same pattern as next_chunk in #98326 & ArrayChunks implementation in #100450.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if we really need a dedicated GuardBack<'a, T> struct for this? The underlying members of GuardBack<'a, T> is the same as Guard<'a, T> with the only difference being the Drop implementation, which makes me think that Guard<'a, T> can be reused for this purpose.
For example, I was taking a look at this discussion on using TypeId, which exists in both core::any and std::any module. Would it be possible (or a good idea) to approach making the guard like so:
// Empty ZST stucts
struct Front {};
struct Back {};
struct Guard<'a, T, D: 'static> {
/// The array to be initialized.
pub array_mut: &'a mut [MaybeUninit<T>],
/// The number of items that have been initialized so far.
pub initialized: usize,
_direction: D
}
impl<T, D: 'static> std::ops::Drop for Guard<'_, T, D> {
fn drop(&mut self) {
match TypeId::of::<D>() {
id if id == TypeId::of::<Front>() => {
// drop impl from Guard<'a, T>
},
id if id == TypeId::of::<Back>() => {
// drop impl from GuardBack<'a, T>
},
_ => {
// this shouldn't happen (mark as unreachable!()?)
},
}
}
}If we are able to reuse Guard<'a, T> like this, I think instead of only having push_unchecked(), we should have two different functions: push_unchecked_front(), which replaces the name of Guard<'a, T>'s push_unchecked(), and push_unchecked_back(), which is what GuardBack<'a, T>'s push_unchecked() does. On a nit, I do think we can give it a better name than Guard<'a,T> (ChunkGuard<'a, T>?), but I don't think that's too important to worry about since this struct is private.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I get the idea and it does reduce the number of structs on paper - but it also pulls in a lot of clever stuff (generics, TypeId, 'static bounds) for something that’s really just an internal guard.
For me, keeping Guard and GuardBack separate keeps the Drop logic dead simple and very obvious - which matters, because that’s the part doing the safety work. One drops 0..n, the other drops len-n..len. You read it once and you’re done.
a4f3496 to
6fc1bec
Compare
Clarify that GuardBack initializes arrays from the end. Restore performance note for iter_next_chunk_erased regarding optimization issues.
|
@joboet ready for review |
Part of #98326
This adds
next_chunk_backtoDoubleEndedIteratoras accepted in ACP #734 (rust-lang/libs-team#734).It optimizes
ArrayChunks::try_rfoldwhich previously required a "double-reverse" when iterating backwards (as noted in a FIXME).Before:
iter.rev().next_chunk()followed bychunk.reverse()After: Direct usage of
iter.next_chunk_back()Implementation details:
DoubleEndedIterator::next_chunk_backas the symmetric reverse counterpart toIterator::next_chunk.GuardBackhelper for correct panic-safe back-to-front array initialization.