Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable Spark SQL tests for Spark 3.5.1 #603

Merged
merged 15 commits into from
Jul 2, 2024
Merged

feat: Enable Spark SQL tests for Spark 3.5.1 #603

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
f4643d6
Enable Spark SQL tests for Spark 3.5.1
andygrove Jun 27, 2024
84cae38
fix conflicts
andygrove Jun 27, 2024
e7cc62c
finish creating diff
andygrove Jun 27, 2024
29a780a
improve docs, update slf4j version
andygrove Jun 28, 2024
a9820f0
improve docs
andygrove Jun 28, 2024
590ad8e
fix diff
andygrove Jun 28, 2024
0559597
fix diff
andygrove Jun 28, 2024
d556eb3
address feedback
andygrove Jun 28, 2024
2d489ef
add IgnoreComet to patch
andygrove Jun 30, 2024
fa4ac9c
update test
andygrove Jun 30, 2024
147c6a2
remove unused imports
andygrove Jun 30, 2024
a109f0d
ignore 2 failing tests
andygrove Jun 30, 2024
7b87ff8
scalastyle
andygrove Jun 30, 2024
345592a
ignore failing parquet metadata tests
andygrove Jun 30, 2024
6e93f4b
add links to tracking issue
andygrove Jul 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/spark_sql_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
matrix:
os: [ubuntu-latest]
java-version: [11]
spark-version: [{short: '3.4', full: '3.4.3'}]
spark-version: [{short: '3.4', full: '3.4.3'}, {short: '3.5', full: '3.5.1'}]
module:
- {name: "catalyst", args1: "catalyst/test", args2: ""}
- {name: "sql/core-1", args1: "", args2: sql/testOnly * -- -l org.apache.spark.tags.ExtendedSQLTest -l org.apache.spark.tags.SlowSQLTest}
Expand Down Expand Up @@ -75,6 +75,7 @@ jobs:
- name: Run Spark tests
run: |
cd apache-spark
rm -rf /root/.m2/repository/org/apache/parquet # somehow parquet cache requires cleanups
ENABLE_COMET=true build/sbt ${{ matrix.module.args1 }} "${{ matrix.module.args2 }}"
env:
LC_ALL: "C.UTF-8"
Expand Down
Loading
Loading