You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I don't think that's possible with the way target-csv currently works. The logic to write to the file, and more importantly here create if it doesn't exist, lives in the process_batch implementation:
My guess is that it's never called for empty streams because the method is only called when a "batch" is full, but that never happens for the stream in question. One fix I can think of is to add an optional early file creation step controlled by a new setting (e.g create_files_for_empty_streams), maybe within the Sink class __init__ method.
I don't think that's possible with the way target-csv currently works. The logic to write to the file, and more importantly here create if it doesn't exist, lives in the
process_batch
implementation:target-csv/target_csv/sinks.py
Lines 103 to 108 in 47a1c76
My guess is that it's never called for empty streams because the method is only called when a "batch" is full, but that never happens for the stream in question. One fix I can think of is to add an optional early file creation step controlled by a new setting (e.g
create_files_for_empty_streams
), maybe within the Sink class__init__
method.https://meltano.slack.com/archives/C069CQNHDNF/p1718436121705219?thread_ts=1718426268.425619&cid=C069CQNHDNF
The text was updated successfully, but these errors were encountered: