How to get columns and change their names in a pyrrow compute expression.

[davlee1972]

Here is a sample pyarrow compute expression used for filtering data:

>>> import pyarrow.compute as pc
>>>
>>> exp = (pc.field("total") > 100) & (pc.field("items") < 10)
>>> exp
<pyarrow.compute.Expression ((total > 100) and (items < 10))>

My odd use case is that I need to modify the expression to:

>>> import pyarrow.compute as pc
>>>
>>> exp = (pc.field("TOTAL") > 100) & (pc.field("ITEMS") < 10)
>>> exp
<pyarrow.compute.Expression ((TOTAL > 100) and (ITEMS < 10))>

Some of my files have lower case column names and some files use upper case column names. I know which files are upper case vs lower case so I need to take a pyarrow compute expression using lower case column names including nested columns in nested conditions and rewrite the expression with all upper case column names.

Is there any way to create something like an abstract syntax tree to get all possible column names and reconstruct the expression with all upper case column names?

[drin]

you probably want to use something like acero (substrait) to do what you want. I can try and figure out how to do it, I don't know it off the top of my head.

docs here (I think): https://arrow.apache.org/docs/python/api/acero.html