Skip to content

feat(core): add MaskExpression POJO and ReadRel projection support#782

Draft
flex-seongmin wants to merge 3 commits intosubstrait-io:mainfrom
flex-team:feat/add-mask-expression
Draft

feat(core): add MaskExpression POJO and ReadRel projection support#782
flex-seongmin wants to merge 3 commits intosubstrait-io:mainfrom
flex-team:feat/add-mask-expression

Conversation

@flex-seongmin
Copy link
Copy Markdown

Summary

Implement the MaskExpression POJO layer for ReadRel projection support in substrait-java.

The Substrait protobuf spec defines Expression.MaskExpression as a field on ReadRel (field 4: projection), but the Java POJO layer had only a TODO comment and no actual implementation. This means any ReadRel projection information was silently dropped during proto-to-POJO conversion, and POJO-constructed plans could not express column projection at the read level.

This change closes that gap by introducing a full POJO model for MaskExpression and wiring it through the existing conversion infrastructure.

Changes

  • MaskExpression.java (new) -- @Value.Enclosing interface with 11 @Value.Immutable inner types (MaskExpr, Select, StructSelect, StructItem, ListSelect, ListSelectItem, ListElement, ListSlice, MapSelect, MapKey, MapKeyExpression) matching the proto definition 1:1
  • MaskExpressionProtoConverter.java (new) -- Dedicated bidirectional converter between io.substrait.proto.Expression.MaskExpression and the POJO types, keeping proto FQN references isolated from the relation converters
  • AbstractReadRel.java -- Replaced TODO comment with Optional<MaskExpression.MaskExpr> getProjection()
  • ProtoRelConverter.java -- Added optionalMaskExpression() helper; all four ReadRel subtypes (NamedScan, LocalFiles, VirtualTableScan, ExtensionTable) now populate projection during proto-to-POJO conversion
  • RelProtoConverter.java -- All four visit(ReadRel) methods now serialize projection back to proto via MaskExpressionProtoConverter.toProto()

Testing

Added 11 roundtrip tests in ReadRelRoundtripTest covering:

  • Simple flat column projection (StructSelect only)
  • Nested struct projection (StructItem with child Select.ofStruct)
  • List element and slice selection (ListSelect)
  • List selection with nested child (ListSelect + child StructSelect)
  • Map key selection (MapSelect with MapKey)
  • Map key expression / wildcard selection (MapKeyExpression)
  • Map selection with nested child (MapSelect + child StructSelect)
  • maintainSingularStruct flag
  • Projection combined with filter
  • Projection on VirtualTableScan and LocalFiles

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 27, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants