-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/transform fields #289
Conversation
… & skip everything else.
private List<Integer> correspondingIndexes; | ||
private final List<Integer> counterIndexes; | ||
protected Map<Featureset, Feature> featureMap; | ||
public Logger logger = LoggerFactory.getLogger(this.getClass().getName()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you make this whole file change be smaller - it is difficult to understand what has actually changed in this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I've requested a couple trivial changes.
@@ -366,6 +366,27 @@ spark.cdm.perfops.ratelimit.target 20000 | |||
#spark.cdm.feature.explodeMap.target.name.key my_map_key | |||
#spark.cdm.feature.explodeMap.target.name.value my_map_value | |||
|
|||
|
|||
#=========================================================================================================== |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we please add an example here on how it will look like? I know we haven't provided examples at other properties, but this is a bit nuanced and may help users understand how it could be used. Thank you in advance!
* Cassandra bump to 5.0-rc1 * Bump Spark to 3.5.2
Supports extracting value from a JSON column in Origin and map it to a specific field on Target
Checklist: