Fix to filter based on dates equal to or later than the start_date

[yamotech]

Description & motivation

In the original code cast(replace(_table_suffix, 'intraday_', '') as int64), larger values than the start_date were being retrieved, resulting in unsuccessful filtering.

select distinct
    _table_suffix
    , cast(replace(_table_suffix, 'intraday_', '') as int64) as original
    , cast(left(replace(_table_suffix, 'intraday_', ''), 8) as int64) as correct
from `{{ target.project }}`.`{{ var('combined_dataset') }}`.`events_*`
limit 1

• original: {{ YYYYMMDD }}{{ property_id }}
• correct: {{ YYYYMMDD }}

Checklist

• I have verified that these changes work locally
• [na] I have updated the README.md (if applicable)
• [na] I have added tests & descriptions to my models (and macros if applicable)
• I have run dbt test and python -m pytest . to validate existing tests

[adamribaudo-velir]

Good catch!

[sippola]

Happy to see this being fixed! I just discovered this issue today when investigating performance issues in dev envs that (attempt to) process limited data via the start_date variable 😄

@adamribaudo-velir is there any estimate of when a release containing this fix would go out?

[adamribaudo-velir]

I can do that tomorrow. You can point your packages.yml to this specific commit in the meantime if you need it sooner.