Skip to content

Commit

Permalink
improve partition_by missing error message
Browse files Browse the repository at this point in the history
  • Loading branch information
MichelleArk committed Sep 25, 2024
1 parent 846eb3f commit 3a9d7ce
Showing 1 changed file with 7 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,14 @@
{{ get_insert_overwrite_sql(source, target, existing) }}
{%- elif strategy == 'microbatch' -%}
{#-- microbatch wraps insert_overwrite, and requires a partition_by config #}
{% set missing_partition_key_microbatch_msg -%}
dbt-spark 'microbatch' incremental strategy requires a `partition_by` config.
Ensure you are using a `partition_by` column that is of grain {{ config.get('batch_size') }}
for microbatch model {{ model.name }}.
{%- endset %}

{%- if not config.get('partition_by') -%}
{{ exceptions.raise_compiler_error("dbt-spark 'microbatch' requires a `partition_by` config") }}
{{ exceptions.raise_compiler_error(missing_partition_key_microbatch_msg) }}
{%- endif -%}
{{ get_insert_overwrite_sql(source, target, existing) }}
{%- elif strategy == 'merge' -%}
Expand Down

0 comments on commit 3a9d7ce

Please sign in to comment.