Skip to content

Conversation

@SFJohnson24
Copy link
Collaborator

@SFJohnson24 SFJohnson24 commented Jan 15, 2026

This PR expands upon #1372 and issues reported with comparing less-precise datetimes.
This PR does a few things:

  1. Incorrect tie-breaking logic - Dates were compared beyond their common precision level causing false positives
    this led to the issue @eljanssens reported with FB0801 "2013-01-23T05:10" > "2013-01-23"

  2. Uncertain date parsing bug - Failed to handle "-" in date strings, causing int("-") conversion errors

  3. removed the logger from is_complete_date() when it fails the first parsing but succeeds on the fallback. This incorrectly stated the parsing failed

  4. I also added test cases for this type of issue. I also fixed some of the date testing-- this testing was incorrect as it compared

 "target": [
                    "1997-07",
                    "1997-07-16",
                    "1997-07-16T19:20:30.45+01:00",
                    "1997-07-16T19:20:30+01:00",
                    "1997-07-16T19:20+01:00",
                ],
                "comparator": [
                    "1997-07",
                    "1997-08-16",
                    "1997-08-16T19:20:30.45+01:00",
                    "1997-08-16T19:20:30+01:00",
                    "1997-08-16T19:20+01:00",
                ],

and

 {
                "target": [
                    "1997-07",
                    "1997-07-16",
                    "1997-07-16T19:20:30.45+01:00",
                    "1997-07-16T19:20:30+01:00",
                    "1997-07-16T19:20+01:00",
                ]
            },
            "1997-07",

All of the target dates are less than or equal to the comparator, given their precision. Same with the second case.

            {
                "target": [
                    "1997-07",
                    "1997-07-16",
                    "1997-07-16T19:20:30.45+01:00",
                    "1997-07-16T19:20:30+01:00",
                    "1997-07-16T19:20+01:00",
                ]
            },
            "1997-07",
            PandasDataset,
            [False, False, False, False, False],

similarly 1997-07 is not greater than any of these as they all have the same date when comparing the least precise

I made test data for FB0801 that has a value in the last row. It is the same date as RFXENDTC but contains a time, it is NOT greater than. If run through current engine, this row is reported. This PR resolves this.
Rule_underscores.json
Datasets2.json
Datasets2.xlsx

@SFJohnson24 SFJohnson24 self-assigned this Jan 15, 2026
Copy link
Collaborator

@RamilCDISC RamilCDISC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR fixes the date comparison in engine. The PR was validated by:

  1. Reviewing the PR for any unwanted code or changes.
  2. Reviewing the PR logic in accordance with AC.
  3. Ensuring all unit and regression testing pass.
  4. Ensuring all related testing is updated.
  5. Manual validation using dev instance of rule editor.
  6. Manual validation using CLI, confirming proper functionality for following operators.
  • date_greater)than
  • date_than_or_equal_to
  • date_less_than
  • date_less_than_or_equal_to
  • date_equal_to
  • date_not_equal_to
  • is_complete_date
  • is_incomplete_date
  • invalid date

@SFJohnson24 SFJohnson24 merged commit 8a66973 into main Jan 19, 2026
13 checks passed
@SFJohnson24 SFJohnson24 deleted the date branch January 19, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants