Skip to content

[Doc] CAST has no explicit TimeType branch (Spark 4.1) #4490

@andygrove

Description

@andygrove

Describe the bug

Spark 4.1 introduced TimeType (HH:mm:ss). Spark 4.1's Cast.scala adds many TimeType arms (StringType <-> TimeType, TimeType <-> TimeType, TimeType -> IntegralType / DecimalType, etc.). CometCast.isSupported has no TimeType arm at all, so every TimeType-involving cast falls into the default case _ => unsupported(fromType, toType) and falls back to Spark.

The fallback is correct, but it is implicit, and getUnsupportedReasons() on CometCast does not enumerate TimeType as one of the unsupported branches, so the auto-generated compatibility guide doesn't mention it.

Surfaced by the cast audit (collection PR queue).

Expected behavior

Either:

  1. Add an explicit TimeType arm to CometCast.isSupported that returns Unsupported(Some("Cast to/from TimeType is not yet supported")), OR
  2. Implement native TimeType cast (depends on TimeType being supported broadly across Comet, tracked by the broader date/time epic [EPIC] Implement all Spark date/time expressions #4418).

The minimum here is the explicit Unsupported arm so the compatibility guide reflects it.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions