Date Comparison with different precision #1544
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR expands upon #1372 and issues reported with comparing less-precise datetimes.
This PR does a few things:
Incorrect tie-breaking logic - Dates were compared beyond their common precision level causing false positives
this led to the issue @eljanssens reported with FB0801 "2013-01-23T05:10" > "2013-01-23"
Uncertain date parsing bug - Failed to handle "-" in date strings, causing int("-") conversion errors
removed the logger from is_complete_date() when it fails the first parsing but succeeds on the fallback. This incorrectly stated the parsing failed
I also added test cases for this type of issue. I also fixed some of the date testing-- this testing was incorrect as it compared
and
All of the target dates are less than or equal to the comparator, given their precision. Same with the second case.
similarly 1997-07 is not greater than any of these as they all have the same date when comparing the least precise
I made test data for FB0801 that has a value in the last row. It is the same date as RFXENDTC but contains a time, it is NOT greater than. If run through current engine, this row is reported. This PR resolves this.
Rule_underscores.json
Datasets2.json
Datasets2.xlsx