Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement Spark-compatible CAST from String to Date #383

Merged
merged 21 commits into from
May 23, 2024
+429 −21
Merged

feat: Implement Spark-compatible CAST from String to Date #383

Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
8b72315
support casting DateType in comet
vidyasankarv May 5, 2024
8e20cb3
Use NaiveDate methods for parsing dates and remove regex
vidyasankarv May 7, 2024
d398c16
remove macro for date parsing
vidyasankarv May 8, 2024
c33c739
compute correct days since epoch.
vidyasankarv May 8, 2024
d4c26ff
Run String to Date test without fuzzy test
vidyasankarv May 8, 2024
1217377
port spark string to date processing logic for cast
vidyasankarv May 10, 2024
a2ce92d
put in fixes for clippy and scalafix issues.
vidyasankarv May 12, 2024
b3ffe9b
handle non digit characters when parsing segement bytes.
vidyasankarv May 13, 2024
245d884
add note on compatability
vidyasankarv May 13, 2024
33f0dfd
add negative tests for String to Date test
vidyasankarv May 14, 2024
e71cb07
simplify byte digit check
vidyasankarv May 14, 2024
736f05d
propagate error correctly when a date cannot be parsed
vidyasankarv May 15, 2024
2e33646
add fuzz test with unsupported date filtering
vidyasankarv May 15, 2024
8bf5f09
add test for string array cast to date
vidyasankarv May 15, 2024
a439490
use UNIX_EPOCH constant from NaiveDateTime
vidyasankarv May 15, 2024
7708a24
fix cargo clippy error - collapse else if
vidyasankarv May 15, 2024
45c25ef
do not run string to date test on spark-3.2
vidyasankarv May 16, 2024
19fa952
do not run string to date test on spark-3.2
vidyasankarv May 17, 2024
2b4c204
add failing fuzz test dates to tests and remove failing fuzz test
vidyasankarv May 20, 2024
9211af5
remove unused date pattern
vidyasankarv May 20, 2024
88af45c
add specific match for casting dictionary to date
vidyasankarv May 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading