Skip to content

Commit

Permalink
feat: Implement Spark-compatible CAST from String to Date (apache#383)
Browse files Browse the repository at this point in the history
* support casting DateType in comet

* Use NaiveDate methods for parsing dates and remove regex

* remove macro for date parsing

* compute correct days since epoch.

* Run String to Date test without fuzzy test

* port spark string to date processing logic for cast

* put in fixes for clippy and scalafix issues.

* handle non digit characters when parsing segement bytes.

* add note on compatability

* add negative tests for String to Date test

* simplify byte digit check

* propagate error correctly when a date cannot be parsed

* add fuzz test with unsupported date filtering

* add test for string array cast to date

* use UNIX_EPOCH constant from NaiveDateTime

* fix cargo clippy error - collapse else if

* do not run string to date test on spark-3.2

* do not run string to date test on spark-3.2

* add failing fuzz test dates to tests and remove failing fuzz test

* remove unused date pattern

* add specific match for casting dictionary to date

(cherry picked from commit a7272b9)
  • Loading branch information
vidyasankarv authored and Huaxin Gao committed May 31, 2024
1 parent 6b34a77 commit f0872e4
Show file tree
Hide file tree
Showing 4 changed files with 429 additions and 21 deletions.
Loading

0 comments on commit f0872e4

Please sign in to comment.