`CREATE OR REPLACE VIEW` preserves other dialects.

[Guosmilesmile]

Feature Request / Improvement

For a view that contains multiple dialects, if we run CREATE OR REPLACE VIEW directly, it will delete the SQL for the other dialects and leave only the latest Spark SQL. Is my understanding correct that this should instead update only the Spark portion of the SQL and preserve the SQL for other engines?

Query engine

Spark

Willingness to contribute

• I can contribute this improvement/feature independently
• I would be willing to contribute this improvement/feature with guidance from the Iceberg community
• I cannot contribute this improvement/feature at this time

[Guosmilesmile]

@singhpk234 @RussellSpitzer @ajantha-bhat I’d like to hear your thoughts on this.

[kevinjqliu]

On the surface level, i think CREATE OR REPLACE VIEW can be treated as DROP VIEW IF EXISTS followed by CREATE VIEW. In fact, this is mentioned in DBX's CREATE VIEW documentation

However, whats created here is an Iceberg view, not a Spark view. Iceberg views can be shared by multiple engines, so CREATE OR REPLACE VIEW shouldn't drop other dialects. Seems like an edge case.

The current view logic might just look at whether a view object exists in the catalog, and not whether the view objects exists for the specific dialect. (I might be wrong here, I didn't double check the code)

[Guosmilesmile]

iceberg/spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java

Lines 654 to 663 in d7aa467

	org.apache.iceberg.view.View view =
	asViewCatalog
	.buildView(buildIdentifier(ident))
	.withDefaultCatalog(currentCatalog)
	.withDefaultNamespace(Namespace.of(currentNamespace))
	.withQuery("spark", sql)
	.withSchema(icebergSchema)
	.withLocation(properties.get("location"))
	.withProperties(props)
	.createOrReplace();

@kevinjqliu I looked into the Spark-related part, and it does only pass SQL in its own dialect. If Iceberg views are meant to support multiple dialects, it would be more reasonable to preserve the dialects from other engines as well—you’re right. The engine layer probably shouldn’t worry about this; instead, the replace view logic in the core module that we call should be adjusted. I’ll take a look and try to figure it out.

Thanks.

[kevinjqliu]

ty! this is a good topic to bring up in the devlist. I was involved with the view spec implementation for spark, maybe there's some context im missing 😄

[Guosmilesmile]

@kevinjqliu I dug a bit deeper. In the original implementation, the replace.drop-dialect.allowed configuration controls whether one dialect is allowed to drop other dialects. So if this config is not enabled, we cannot perform a replace.

However, with the current approach, we end up having to duplicate a view into multiple copies just to maintain separate engines, and we can’t reuse the same view.

I opened a PR to preserve the dialect of the version engine for replace scenarios.