Version 0.1.1

✨NEW

• Add a composite transform (WriteToDynamoDB) that writes records to a DynamoDB table with help of the batch_writer of the boto3 package.
  • The batch writer will automatically handle buffering and sending items in batches. In addition, it will also automatically handle any unprocessed items and resend them as needed.
• Provide an option that handles duplicate records
  • dedup_pkeys - List of keys to be used for deduplicating items in buffer.
• Create a dedicated pipeline option (DynamoDBOptions) that reads AWS related values (e.g. aws_access_key_id) from pipeline arguments.
• Implement a metric object that records the total counts.
• Add unit and integration testing cases. The moto and localstack-utils are used for unit and integration testing respectively.
• Integrate with GitHub Actions by adding workflows for testing, documentation and release management.