5.1.4

Key Highlights

• Bug fix: Any run started with a previousRunId that is not found in the cdm_run_info table (for whatever reason), will be executed as a fresh new run instead of doing nothing.

Jar file can also be found in the packages section here.

5.1.3

Key Highlights

• Bug fix: Fixed connection issue caused when using different types of origin and target clusters (e.g. Cassandra/DSE with host/port and Astra with SCB).

Jar file can also be found in the packages section here.

5.1.2

Key Highlights

• Bug fix: SCB file on some Spark worker nodes may get deleted before the connection is established, which may cause connection exception on that worker node. Added a static async SCB delete delay to address such issues.

Jar file can also be found in the packages section here.

5.1.1

Key Highlights

• Writetime filter has been fixed to work correctly when used with custom writetimestamp (bug fixed)
• Removed deprecated properties printStatsAfter and printStatsPerPart. Run metrics should now be tracked using the trackRun feature instead.

Jar file can also be found in the packages section here.

5.1.0

Key Highlights

• Improves metrics output by producing stats labels in an intuitive and consistent order
• Refactored JobCounter by removing any references to thread or global as CDM operations are now isolated within partition-ranges (parts). Each such part is then parallelly processed and aggregated by Spark.

Jar file can also be found in the packages section here.

5.0.0

Key Highlights

• CDM refactored to be fully Spark Native and more performant when deployed on a multi-node Spark Cluster
• trackRun feature has been expanded to record run-info for each part in the CDM_RUN_DETAILS table. Along with granular metrics, this information can be used to troubleshoot any unbalanced problematic partitions.
• This release has feature parity with 4.x release and is also backword compatible while adding the above mentioned improvements. However, we are upgrading it to 5.x as its a major rewrite of the code to make it Spark native.

Jar file can also be found in the packages section here.

4.6.1

Key Highlights

• Make trackRun feature work on all versions of Cassandra/DSE by replacing the IN clause on cdm_run_details table.
• Updated README docs.

Jar file can also be found in the packages section here.

4.6.0

Key Highlights

• Allow using Collections and/or UDTs for ttl & writetime calculations. This is specifically helpful in scenarios where the only non-key columns are Collections and/or UDTs.

Jar file can also be found in the packages section here.

4.5.1

Key Highlights

• Made CDM generated SCB unique & much short-lived when using the TLS option to connect to Astra more securely.

Jar file can also be found in the packages section here.

4.5.0

Key Highlights

• Upgraded to use log4j 2.x and included a template properties file that will help separate general logs from CDM class specific logs including a separate log for rows identified by DiffData (Validation) errors.
• Upgraded to use Spark 3.5.3

Jar file can also be found in the packages section here.