v7.0.0

7.0.0 (2024-12-18)

⚠ BREAKING CHANGES

• Exclude views from output of find tables (#1355)

Features

• Add UUID support (#1367) (4646757)
• Auto populate -pks when not provided by user (#1324) (6119c18)
• Exclude views from output of find tables (#1355) (eafdc93)
• Prevent column validation exceptions caused by Oracle CLOB JSON columns (#1365) (b20b4dd)
• Support customer defined api endpoint for BigQuery and Spanner (#1340) (c88e752)

Bug Fixes

• 1261 pg custom query without creating view (#1353) (f25fe80)
• catch InvalidArg when checking maybe_secret (#1332) (2924202)
• Fix issue where long rows were being truncated in raw query command (#1362) (97cfdbd)

Documentation

• Refine connection secret support section (#1358) (78c79ef)
• Update docs/connections.md and samples/oracle/README.md (#1373) (62d0718)

v6.4.0

6.4.0 (2024-11-12)

Features

• user defined input run_id via config (#1310) (783d1b0)

Bug Fixes

• Fix --comp-fields bug losing option value before validation (#1323) (8813366)
• Hardcode Oracle max_identifier_length=128, this is a hack but a necessary evil (#1316) (4c54031)
• Issue 1312 incorrect status code (#1317) (ade6d79)
• Issue 1325 (#1328) (dfe8b8b)

Documentation

• Add a sample script for VM automation of partitioning (#1307) (75089b1)

v6.3.0

6.3.0 (2024-10-24)

Features

• Add support for -comp-fields=*, which also adds support for --exclude-columns (#1291) (f181377)
• Added ORC and PARQUET support among source tests (#1298) (e102d1e)
• Partition Custom Queries (#1289) (7e947d5)
• Support row validation of boolean columns (#1284) (c1de5f3)

Bug Fixes

• Add proper handling for GCPSecretManager exceptions (#1285) (8028836)
• Combiner did not take boolean datatypes into account (#1290) (84824db)
• Correctly cater for Oracle FLOAT in custom-query validations (#1304) (fbe9d23)

v6.2.0

6.2.0 (2024-09-27)

Features

• Enhance --tables-list to support schema.* shorthand for find-tables command (#1265) (cbd9bbb)

Bug Fixes

• big query run id message (#1263) (09659e0)
• cast count to bigint when generating partitions (#1281) (a2861ca)
• Explicit set the parameter for line terminator on converting datafram… (#1269) (b20b2b5), closes #1248
• Oracle validations of tables with hyphen in name (#1245) (4adaa79)
• Set RAND() as default for DB2's random function (#1246) (d01a40b)
• Update Teradata rtrim() to account for tabs and other whitespace (#1273) (db7e9d3)
• Using datetime constants in filters (#1259) (44863bd)

Documentation

• Add comments on both verbose and log-level parameters, also alter the… (#1257) (35caf98)

v6.1.0

6.1.0 (2024-08-27)

Features

• Add Oracle TIMESTAMP WITH LOCAL TIME ZONE support (#1238) (1e9f458)
• Add TIME data type support for Teradata, BigQuery and SQL Server (#1229) (ab7007b)
• Adding exclude_columns flag for row validation hash (#1243) (a1fd616)
• Auto split row concat/hash validations when many columns (#1233) (ae9b72d)
• BigQuery result handler logs textual output at DEBUG instead of INFO (#1240) (a9aafa2)

Bug Fixes

• Adds automatic RTRIM for custom query row validations for Teradata string comparison fields (#1230) (5c1a2be)
• Bug for generate partitions with dates with different column names for PKs (#1231) (5f51653)
• Fixing multiple function call, to get schema in custom-query validation for Hive (#1180) (a584e5b)
• support generate partitions for one row per partition (#1241) (4099a29)
• TD to BQ - Support hash row validation with Latin and Unicode Characters in Strings (#1226) (e1b24ef)
• TD to BQ generate partitions with date PKs (#1220) (a03f3b0)

Documentation

• Update Cloud Functions sample to support Cloud Scheduler job triggering (#1228) (3a7c164)

v6.0.0

6.0.0 (2024-08-06)

⚠ BREAKING CHANGES

• Failing validations of strings with extended ascii characters (#1184)

Features

• Add TLS options to PostgreSQL connections add (#1199) (5506680)
• Enhance PostgreSQL connections add documentation (#1200) (f179f5a)
• generate-table-partitions write multiple partitions to a single yaml file (#1178) (59a9a18)

Bug Fixes

• Apply RTRIM on string column when generating partitions with -tsp (#1182) (9dcaad1)
• Close source and target connections after executing a validation (#1197) (3dc9fa7)
• Ensure BigQuery queries are executed in UTC (#1174) (a77cdd9)
• Ensure Teradata OutofBound dates don't affect other date columns (#1219) (e02609d)
• Failing validations of strings with extended ascii characters (#1184) (c8ac146)
• Issue with GoogleSQL regarding dates before 1000CE (#1186) (6107d9c)
• More robust exit on DataValidation() (#1206) (13ec3ee)
• Oracle cast(decimal to string) caters for non-zero values <1 (#1204) (fa44466)
• Oracle INTERVAL exception in validate schema (#1215) (f133f73)
• Support char comparison fields in Teradata (#1203) (dc19580)
• Support DATE '0001-01-01' comparison fields (#1208) (cca0c4f)

Documentation

• Fix some typos in Cloud Run readme (#1188) (c8cd0de)
• Tweaks to Cloud Functions sample (#1176) (7a49a95)

v5.1.1

5.1.1 (2024-06-12)

Documentation

• Update generate-partitions flags (#1168) (5a747c2)

v5.1.0

5.1.0 (2024-06-11)

Features

• Add a workaround for a Snowflake IN list limitation (#1152) (16b979e)
• Support --trim-string-pks flag for padded string semantics (#1166) (a81f396)
• Support GCS custom query files (#1155) (e3fe3d1)

Bug Fixes

• Fixes bug in get_max_in_list_size (#1158) (973e6b6)
• Removing t0 alias from column name, while getting schema from query. Adding Integration test for Hive Custom-Query (#1164) (74a14af)
• Support PKs with different casing for generate-partitions (#1142) (021ce75)
• Update to support up to 10K partitions (#1139) (210c352)

v5.0.0

5.0.0 (2024-05-21)

⚠ BREAKING CHANGES

• Support for GCS config paths decoupled from environment variables (#1129)
• Filters not working correctly in Snowflake (#1126)

Features

• Add support for random row sampling on binary id columns (#1135) (c3d2155)
• Control Teradata decimal format when cast to string (#1138) (e68e2a6)
• Support for GCS config paths decoupled from environment variables (#1129) (72e41b7)

Bug Fixes

• Filters not working correctly in Snowflake (#1126) (9845643)
• Fix casting from binary to string on Snowflake & BigQuery (#1113) (4f5ae81)
• Issue 1127 configs dir fails with more than 40 files (#1130) (15c81cf)
• Teradata's ValueError after large timestamp epoch second handling (#1121) (ee8d6da)

Documentation

• Add custom-query code snippet for Cloud Run sample documentation (#1124) (93bb64f)
• Distributed DVT Cloud Run Jobs sample (#1133) (f51f327)

v4.5.0

4.5.0 (2024-03-18)

Features

• Support GCS files in configs list command (#1108) (b49e1c3)

Bug Fixes

• Add table names to report results when source and/or target dataframes are empty (#1104) (812ed62)
• Fixes issue casting Snowflake decimal with scale>0 to string (#1110) (34446a4)
• force cast for aggregates (#1114) (44b60cf)
• Teradata large timestamp handling (#1117) (842d8b7)