Skip to content

feat(parquet): add StatisticsConverter::from_column_index#9687

Open
kumarUjjawal wants to merge 1 commit intoapache:mainfrom
kumarUjjawal:feat/statistical_converter_nested_fields
Open

feat(parquet): add StatisticsConverter::from_column_index#9687
kumarUjjawal wants to merge 1 commit intoapache:mainfrom
kumarUjjawal:feat/statistical_converter_nested_fields

Conversation

@kumarUjjawal
Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

StatisticsConverter::try_new resolves columns through parquet_column(), which only supports top-level non-nested columns. Because of that, callers that already know the Parquet leaf index for a nested field cannot use StatisticsConverter to read statistics for that field.

This blocks reading stats for fields inside structs.

What changes are included in this PR?

  • add StatisticsConverter::from_column_index(...)
  • keep StatisticsConverter::try_new(...) behavior unchanged
  • update docs to explain when to use each constructor
  • add a test that writes a struct column and reads statistics for each child
    field through the new API

Are these changes tested?

Yes

Are there any user-facing changes?

Yes.

This adds a new public constructor: StatisticsConverter::from_column_index(...)

There are no breaking changes.

@adriangb
Copy link
Copy Markdown
Contributor

I think @friendlymatthew is also working on this, maybe you can coordinate efforts?

@alamb
Copy link
Copy Markdown
Contributor

alamb commented Apr 10, 2026

There is this one from @friendlymatthew

it would be super helpful to combine efforts (e.g. by reviewing the PRs together) and pinging me when you think they are ready

@kumarUjjawal
Copy link
Copy Markdown
Author

I have reached out to @friendlymatthew for this, I will get back to you once we decide how to proceed forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

parquet Changes to the parquet crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StatisticsConverter does not support nested fields (e.g. struct fields)

3 participants