Skip to content

Commit

Permalink
partition_by required for dbt-spark microbatch
Browse files Browse the repository at this point in the history
  • Loading branch information
MichelleArk committed Sep 24, 2024
1 parent 287531c commit 835cb47
Showing 1 changed file with 4 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,10 @@
{#-- insert statements don't like CTEs, so support them via a temp view #}
{{ get_insert_overwrite_sql(source, target, existing) }}
{%- elif strategy == 'microbatch' -%}
{#-- insert statements don't like CTEs, so support them via a temp view #}
{#-- microbatch wraps insert_overwrite, and requires a partition_by config #}
{%- if not config.get('partition_by') -%}
{{ exceptions.raise_compiler_error("dbt-spark 'microbatch' requires a `partition_by` config") }}
{%- endif -%}
{{ get_insert_overwrite_sql(source, target, existing) }}
{%- elif strategy == 'merge' -%}
{#-- merge all columns for datasources which implement MERGE INTO (e.g. databricks, iceberg) - schema changes are handled for us #}
Expand Down

0 comments on commit 835cb47

Please sign in to comment.