-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add Union encoding documentation #9102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
I tested union encoding myself, and the behaviour that I observed differed from the original proposal in #8828. I based my examples on the actual behaviour to the best of my knowledge. Please let me know if something is wrong. |
|
@friendlymatthew -- can you help review this? |
alamb
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks -- I went over this carefully and I think it looks correct and really nicely explained. Thank you @EduardAkhmetshin
cc @Jefffrey
| /// ## Union Ordering | ||
| /// | ||
| /// Values of the same type are ordered according to the ordering of that type. | ||
| /// Values of different types are ordered by their type id. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good to mention reversing via negating the type_id here:
Lines 1742 to 1747 in 2507946
| let type_id_byte = if opts.descending { | |
| !(type_id as u8) | |
| } else { | |
| type_id as u8 | |
| }; | |
| data[*offset] = type_id_byte; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since I am trying to prepare the 57.2.0 I took the liberty of making this change directly to this PR in 08efd42. If I got it wrong or you would like other changes, I will be happy to make a follow on PR
|
Thanks again @EduardAkhmetshin @Jefffrey and @friendlymatthew |
Which issue does this PR close?
What changes are included in this PR?
Documentation on union types encoding in https://arrow.apache.org/rust/arrow_row/struct.RowConverter.html.
Are these changes tested?
Yes.
Are there any user-facing changes?
Yes. https://arrow.apache.org/rust/arrow_row/struct.RowConverter.html will get updated.