Skip to content

Commit

Permalink
Updated spark 2.3.x and spark 2.4.x versions to test against them. (#431
Browse files Browse the repository at this point in the history 
)
  • Loading branch information
@alfonsorr
alfonsorr authored Feb 28, 2024
1 parent f873b0e commit be8f352
Show file tree
Hide file tree
Showing 5 changed files with 65 additions and 33 deletions.
20 changes: 20 additions & 0 deletions .github/workflows/matrix_includes.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,31 @@
"scope": "test",
"isRelease": "-"
},
{
"spark": "3.3.2",
"scope": "test",
"isRelease": "-"
},
{
"spark": "3.3.3",
"scope": "test",
"isRelease": "-"
},
{
"spark": "3.3",
"scope": "test-uploadReport",
"isRelease": "release"
},
{
"spark": "3.4.0",
"scope": "test",
"isRelease": "-"
},
{
"spark": "3.4.1",
"scope": "test",
"isRelease": "-"
},
{
"spark": "3.4",
"scope": "test-uploadReport",
Expand Down
8 changes: 6 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,12 @@ following spark versions.
| 3.2.2 | 2.12 / 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.12/0.0.7) |
| 3.3.0 | 2.12 / 2.13 | ✅ | - |
| 3.3.1 | 2.12 / 2.13 | ✅ | - |
| 3.3.2 | 2.12 / 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.12/0.0.7) |
| 3.4.0 | 2.12 / 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.12/0.0.7) |
| 3.3.2 | 2.12 / 2.13 | ✅ | - |
| 3.3.3 | 2.12 / 2.13 | ✅ | - |
| 3.3.4 | 2.12 / 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.12/0.0.7) |
| 3.4.0 | 2.12 / 2.13 | ✅ | - |
| 3.4.1 | 2.12 / 2.13 | ✅ | - |
| 3.4.2 | 2.12 / 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.12/0.0.7) |



Expand Down
6 changes: 3 additions & 3 deletions build.sbt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ val sparkDefaultShortVersion = "3.4"
val spark30Version = "3.0.3"
val spark31Version = "3.1.3"
val spark32Version = "3.2.3"
val spark33Version = "3.3.2"
val spark34Version = "3.4.0"
val spark33Version = "3.3.4"
val spark34Version = "3.4.2"

val versionRegex = """^(.*)\.(.*)\.(.*)$""".r
val versionRegexShort = """^(.*)\.(.*)$""".r
Expand All @@ -23,7 +23,7 @@ val parserSparkVersion: String => String = {
case versionRegexShort("3", "2") => spark32Version
case versionRegexShort("3", "3") => spark33Version
case versionRegexShort("3", "4") => spark34Version
case versionRegex(a, b, c) => s"$a.$b.$c"
case versionRegex("3", b, c) => s"3.$b.$c"
}

val long2ShortVersion: String => String = { case versionRegex(a, b, _) =>
Expand Down
8 changes: 4 additions & 4 deletions docs/docs/docs/api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Here you can see the whole doric API for every supported spark version:
| 3.1.3 | 2.12 | [doric_3-1_2.12](spark-3.1/scala-2.12/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-1_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-1_2.12/@STABLE_VERSION@) |
| 3.2.3 | 2.12 | [doric_3-2_2.12](spark-3.2/scala-2.12/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.12/@STABLE_VERSION@) |
| 3.2.3 | 2.13 | [doric_3-2_2.13](spark-3.2/scala-2.13/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.13/@STABLE_VERSION@) |
| 3.3.2 | 2.12 | [doric_3-3_2.12](spark-3.3/scala-2.12/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.12/@STABLE_VERSION@) |
| 3.3.2 | 2.13 | [doric_3-3_2.13](spark-3.3/scala-2.13/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.13/@STABLE_VERSION@) |
| 3.4.0 | 2.12 | [doric_3-3_2.12](spark-3.4/scala-2.12/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.12/@STABLE_VERSION@) |
| 3.4.0 | 2.13 | [doric_3-3_2.13](spark-3.4/scala-2.13/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.13/@STABLE_VERSION@) |
| 3.3.4 | 2.12 | [doric_3-3_2.12](spark-3.3/scala-2.12/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.12/@STABLE_VERSION@) |
| 3.3.4 | 2.13 | [doric_3-3_2.13](spark-3.3/scala-2.13/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.13/@STABLE_VERSION@) |
| 3.4.2 | 2.12 | [doric_3-3_2.12](spark-3.4/scala-2.12/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.12/@STABLE_VERSION@) |
| 3.4.2 | 2.13 | [doric_3-3_2.13](spark-3.4/scala-2.13/) | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.13/@STABLE_VERSION@) |
56 changes: 32 additions & 24 deletions docs/docs/docs/quickstart.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,31 +31,39 @@ Doric is committed to use the most modern APIs first.
* The latest experimental version of doric is @VERSION@.
* Doric is compatible with the following Spark versions:

| Spark | Scala | Tested | doric |
|:------:|:-----:|:------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| Spark | Scala | Tested | doric |
|:------------------:|:-----:|:------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| 2.4.x (Deprecated) | 2.11 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_2-4_2.11)](https://mvnrepository.com/artifact/org.hablapps/doric_2-4_2.11/0.0.7) |
| 3.0.0 | 2.12 | ✅ | - |
| 3.0.1 | 2.12 | ✅ | - |
| 3.0.2 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-0_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-0_2.12/@STABLE_VERSION@) |
| 3.1.0 | 2.12 | ✅ | - |
| 3.1.1 | 2.12 | ✅ | - |
| 3.1.2 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-1_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-1_2.12/@STABLE_VERSION@) |
| 3.2.0 | 2.12 | ✅ | - |
| 3.2.0 | 2.13 | ✅ | - |
| 3.2.1 | 2.12 | ✅ | - |
| 3.2.1 | 2.13 | ✅ | - |
| 3.2.2 | 2.12 | ✅ | - |
| 3.2.2 | 2.13 | ✅ | - |
| 3.2.3 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.12/@STABLE_VERSION@) |
| 3.2.3 | 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.13/@STABLE_VERSION@) |
| 3.3.0 | 2.12 | ✅ | - |
| 3.3.0 | 2.13 | ✅ | - |
| 3.3.1 | 2.12 | ✅ | - |
| 3.3.1 | 2.13 | ✅ | - |
| 3.3.2 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.12/@STABLE_VERSION@) |
| 3.3.2 | 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.13/@STABLE_VERSION@) |
| 3.4.0 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.12/@STABLE_VERSION@) |
| 3.4.0 | 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.13/@STABLE_VERSION@) |
| 3.0.0 | 2.12 | ✅ | - |
| 3.0.1 | 2.12 | ✅ | - |
| 3.0.2 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-0_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-0_2.12/@STABLE_VERSION@) |
| 3.1.0 | 2.12 | ✅ | - |
| 3.1.1 | 2.12 | ✅ | - |
| 3.1.2 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-1_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-1_2.12/@STABLE_VERSION@) |
| 3.2.0 | 2.12 | ✅ | - |
| 3.2.0 | 2.13 | ✅ | - |
| 3.2.1 | 2.12 | ✅ | - |
| 3.2.1 | 2.13 | ✅ | - |
| 3.2.2 | 2.12 | ✅ | - |
| 3.2.2 | 2.13 | ✅ | - |
| 3.2.3 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.12/@STABLE_VERSION@) |
| 3.2.3 | 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-2_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-2_2.13/@STABLE_VERSION@) |
| 3.3.0 | 2.12 | ✅ | - |
| 3.3.0 | 2.13 | ✅ | - |
| 3.3.1 | 2.12 | ✅ | - |
| 3.3.1 | 2.13 | ✅ | - |
| 3.3.2 | 2.12 | ✅ | - |
| 3.3.2 | 2.13 | ✅ | - |
| 3.3.3 | 2.12 | ✅ | - |
| 3.3.3 | 2.13 | ✅ | - |
| 3.3.4 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.12/@STABLE_VERSION@) |
| 3.3.4 | 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-3_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-3_2.13/@STABLE_VERSION@) |
| 3.4.0 | 2.12 | ✅ | - |
| 3.4.0 | 2.13 | ✅ | - |
| 3.4.1 | 2.12 | ✅ | - |
| 3.4.1 | 2.13 | ✅ | - |
| 3.4.2 | 2.12 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.12)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.12/@STABLE_VERSION@) |
| 3.4.2 | 2.13 | ✅ | [![Maven Central](https://img.shields.io/maven-central/v/org.hablapps/doric_3-4_2.13)](https://mvnrepository.com/artifact/org.hablapps/doric_3-4_2.13/@STABLE_VERSION@) |


__Import statements__
Expand Down

0 comments on commit be8f352

Please sign in to comment.