-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: bump spark version to 3.4.3 #292
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We only have diff for Spark 3.4.2. To build and test with Spark 3.4.3, you need to prepare diff for Spark 3.4.3.
fe4b923
to
f6dcdd7
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #292 +/- ##
=========================================
Coverage 34.18% 34.18%
Complexity 851 851
=========================================
Files 116 116
Lines 38570 38570
Branches 8531 8531
=========================================
Hits 13187 13187
Misses 22612 22612
Partials 2771 2771 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (pending ci)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
initialSessionOptions: java.util.HashMap[String, String]) = { | ||
this(sc, None, None, | ||
- SparkSession.applyExtensions( | ||
+ SparkSession.applyExtensions(sc, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the most significant change in this PR is this Spark code diff. Do you notice any obvious change to 3.4.2 diffs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me as I quickly look through the 3.4.3 diffs. But wondering if @huaxingao finds any changes between 3.4.2 and 3.4.3 diffs we should give notice.
@viirya
|
Okay, that's great. I thought that there might be some conflicts between 3.4.2 and 3.4.3 tests, but seems we don't have much changes between the two patch versions. |
* build: bump spark version to 3.4.3 * add 3.4.4 diff * fix 3.4.3.diff --------- Co-authored-by: Huaxin Gao <[email protected]>
Which issue does this PR close?
Closes #.
Rationale for this change
What changes are included in this PR?
How are these changes tested?