-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Use enum to represent CAST eval_mode in expr.proto #415
Changes from 4 commits
20770fb
f48bd93
5e6a35c
0b76c4a
773d8d1
8184242
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -233,12 +233,20 @@ message Remainder { | |
DataType return_type = 4; | ||
} | ||
|
||
enum EvalMode { | ||
LEGACY = 0; | ||
TRY = 1; | ||
ANSI = 2; | ||
} | ||
|
||
message Cast { | ||
Expr child = 1; | ||
DataType datatype = 2; | ||
string timezone = 3; | ||
// LEGACY, ANSI, or TRY | ||
string eval_mode = 4; | ||
// Depreciateid: LEGACY, ANSI, or TRY - preserved for backward compatibity | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No need for backwards compatibility here since we have not published a release yet There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed |
||
string eval_mode_string= 4; // for backward compatibility | ||
EvalMode eval_mode = 5; // New enum field | ||
|
||
} | ||
|
||
message Equal { | ||
|
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -19,6 +19,8 @@ | |||||||
|
||||||||
package org.apache.comet.serde | ||||||||
|
||||||||
import java.util.Locale | ||||||||
|
||||||||
import scala.collection.JavaConverters._ | ||||||||
|
||||||||
import org.apache.spark.internal.Logging | ||||||||
|
@@ -525,6 +527,18 @@ object QueryPlanSerde extends Logging with ShimQueryPlanSerde { | |||||||
* @return | ||||||||
* The protobuf representation of the expression, or None if the expression is not supported | ||||||||
*/ | ||||||||
|
||||||||
def stringToEvalMode(evalModeStr: String): ExprOuterClass.EvalMode = | ||||||||
evalModeStr.toUpperCase(Locale.ROOT) match { | ||||||||
case "LEGACY" => ExprOuterClass.EvalMode.LEGACY | ||||||||
case "TRY" => ExprOuterClass.EvalMode.TRY | ||||||||
case "ANSI" => ExprOuterClass.EvalMode.ANSI | ||||||||
case _ => | ||||||||
throw new IllegalArgumentException( | ||||||||
"Invalid eval mode" | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be good to include the invalid value in the error message There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have updated the code as follows. 5 def stringToEvalMode(evalModeStr: String): ExprOuterClass.EvalMode =
4 evalModeStr.toUpperCase(Locale.ROOT) match {
3 case "LEGACY" => ExprOuterClass.EvalMode.LEGACY
2 case "TRY" => ExprOuterClass.EvalMode.TRY
1 case "ANSI" => ExprOuterClass.EvalMode.ANSI
536 case invalid =>
1 throw new IllegalArgumentException(
2 s"Invalid eval mode '$invalid' "
3 ) // Assuming we want to catch errors strictly
4 } |
||||||||
) // Assuming we want to catch errors strictly | ||||||||
} | ||||||||
|
||||||||
def exprToProto( | ||||||||
expr: Expression, | ||||||||
input: Seq[Attribute], | ||||||||
|
@@ -535,12 +549,14 @@ object QueryPlanSerde extends Logging with ShimQueryPlanSerde { | |||||||
childExpr: Option[Expr], | ||||||||
evalMode: String): Option[Expr] = { | ||||||||
val dataType = serializeDataType(dt) | ||||||||
val evalModeEnum = stringToEvalMode(evalMode) // Convert string to enum | ||||||||
|
||||||||
if (childExpr.isDefined && dataType.isDefined) { | ||||||||
val castBuilder = ExprOuterClass.Cast.newBuilder() | ||||||||
castBuilder.setChild(childExpr.get) | ||||||||
castBuilder.setDatatype(dataType.get) | ||||||||
castBuilder.setEvalMode(evalMode) | ||||||||
// castBuilder.setEvalMode(evalMode) | ||||||||
andygrove marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
castBuilder.setEvalMode(evalModeEnum) // Set the enum in protobuf | ||||||||
|
||||||||
val timeZone = timeZoneId.getOrElse("UTC") | ||||||||
castBuilder.setTimezone(timeZone) | ||||||||
|
@@ -1207,7 +1223,7 @@ object QueryPlanSerde extends Logging with ShimQueryPlanSerde { | |||||||
.newBuilder() | ||||||||
.setChild(e) | ||||||||
.setDatatype(serializeDataType(IntegerType).get) | ||||||||
.setEvalMode("LEGACY") // year is not affected by ANSI mode | ||||||||
.setEvalMode(stringToEvalMode("LEGACY")) // year is not affected by ANSI mode | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can just use the enum directly here rather than convert from string.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thank you for another great suggestion. |
||||||||
.build()) | ||||||||
.build() | ||||||||
}) | ||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can make use of the generated protobuf enum here rather than hard-code numberic values. This would also allow us to remove the "other" error handling case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the suggestion, Andy.
I have made changes to
planner.rs
as per suggestion.