Skip to content

[Bug] next_day trims whitespace from dayOfWeek; Spark does not #4450

@andygrove

Description

@andygrove

Describe the bug

next_day(date, dayOfWeek) in Comet trims leading/trailing whitespace from the dayOfWeek argument before matching. Spark's DateTimeUtils.getDayOfWeekFromString does not trim, so values like ' MO ' succeed in Comet but Spark either returns NULL (non-ANSI) or throws (ANSI).

The trim happens in datafusion-spark::SparkNextDay.

Steps to reproduce

SELECT next_day(date('2024-01-01'), ' MO ');
-- Spark: NULL (or throws under ANSI)
-- Comet: 2024-01-08

Expected behavior

Do not trim; match Spark's character-for-character matching.

Additional context

Surfaced by the date/time audit (#4448). Reproducer captured as an ignored test in spark/src/test/resources/sql-tests/expressions/datetime/next_day.sql; remove the ignore(...) when this is fixed.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions