v2.0.0

2.0.0 (2022-09-29)

What's Changed

⚠️ 🚨 BREAKING COMPATIBILITY: Python 3.8 SUPPORT REMOVED 🚨 ⚠️

• [target-core] Move the core features and functions in common shared target-core package by @ome9ax in #35

All the core stream processing functionalities are combined into target-core.

target-core core functionalities

• The stream processing library is using asyncio.to_thread introduced in Python 3.9.
• Better isolation architecture comes now by design between singer stream protocol and output custom processing. This opens for more native processing modularity and flexibility (API, S3, ...).
• Uses sys.stdin.buffer input reader over sys.stdin for more efficient input stream management.

target-s3-jsonl changes

• version ">=2.0" developments will continue under Python 3.9 and above.
• version "~=1.0" will keep living under the legacy-v1 branch.
• Optimised memory and storage management: files are uploaded asynchronously and deleted on the fly, no longer all at once at the end.

Config file updates

• changes (those will be automatically replaced during the deprecation period for backward compatibility):
  • path_template replaces naming_convention (deprecated). Few changes as well in the path_template syntax:
    • {date_time} replaces {timestamp} (deprecated).
    • {date_time:%Y%m%d} replaces {date} (deprecated).
  • work_dir replaces temp_dir (deprecated).
• New option file_size for file partitioning by size limit. The path_template must contain a part section for the part number. Example "path_template": "{stream}_{date_time:%Y%m%d_%H%M%S}_part_{part:0>3}.json".

Full Changelog: 1.2.2...2.0.0