Skip to content

Spark: Add regression test for MERGE struct nullification#16326

Open
yadavay-amzn wants to merge 1 commit into
apache:mainfrom
yadavay-amzn:fix/16246-merge-struct-nullification
Open

Spark: Add regression test for MERGE struct nullification#16326
yadavay-amzn wants to merge 1 commit into
apache:mainfrom
yadavay-amzn:fix/16246-merge-struct-nullification

Conversation

@yadavay-amzn
Copy link
Copy Markdown
Contributor

@yadavay-amzn yadavay-amzn commented May 14, 2026

Closes #16246.

Summary

Adds a regression test verifying that MERGE INTO preserves non-nullable children of a nullable struct column when updating other columns.

Context

Issue #16246 reported that MERGE INTO silently nullified non-nullable struct children on Iceberg 1.5.2 / Spark 3.3. The bug is already fixed on main (likely resolved by schema handling improvements in later versions). This test prevents reintroduction.

Test

testMergePreservesNullableStructWithRequiredChildren:

  1. Creates a table with schema id INT, status STRING, info STRUCT<type:STRING NOT NULL, attr:STRING NOT NULL>
  2. Inserts a row with non-null struct data
  3. Performs MERGE INTO updating only the status column
  4. Asserts struct children remain non-null

@github-actions github-actions Bot added the spark label May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Silent data nullification during MERGE when nullable struct contains non-nullable children

1 participant