Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update
MicrobatchBuilder.build_start_time
to not create start times…
… that equal the checkpoint Previously if the checkpoint provided to `build_start_time` was at the truncation point for the related batch size, then the returned `start_time` would be _the same_ as the checkpoint. For example if the checkpoint was "2024-09-05 00:00:00", and the batch size was `day`, then the returend `start_time` would be "2024-09-05 00:00:00". This is problematic because then the would be no batch created when running `build_batches`. Or, prior to 12bb2ca, you'd get one batch with a filter like `event_time >= 2024-09-05 00:00:00 AND event_time < 2024-09-05 00:00:00` which is impossible to satisfy. The change in this PR makes it so that if the checkpoint is at the truncation point, then the start time will be guaranteed to move back by one batch period. That is, following the same example, "2024-09-04 00:00:00" would be returned.
- Loading branch information