Skip to content

Commit

Permalink
add prefix and suffix arguments to star macro (#436)
Browse files Browse the repository at this point in the history
* add prefix and suffix to star macro

* add tests

* snowflake casing?
  • Loading branch information
fivetran-jamie authored Nov 26, 2021
1 parent 6ed3130 commit cffe8d5
Show file tree
Hide file tree
Showing 5 changed files with 27 additions and 5 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -726,7 +726,7 @@ group by 1,2,3
```

#### star ([source](macros/sql/star.sql))
This macro generates a list of all fields that exist in the `from` relation, excluding any fields listed in the `except` argument. The construction is identical to `select * from {{ref('my_model')}}`, replacing star (`*`) with the star macro. This macro also has an optional `relation_alias` argument that will prefix all generated fields with an alias.
This macro generates a list of all fields that exist in the `from` relation, excluding any fields listed in the `except` argument. The construction is identical to `select * from {{ref('my_model')}}`, replacing star (`*`) with the star macro. This macro also has an optional `relation_alias` argument that will prefix all generated fields with an alias (`relation_alias`.`field_name`). The macro also has optional `prefix` and `suffix` arguments, which will be appropriately concatenated to each field name in the output (`prefix` ~ `field_name` ~ `suffix`).

**Usage:**
```sql
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
prefix_field_1_suffix,prefix_field_2_suffix,prefix_field_3_suffix
a,b,c
d,e,f
g,h,i
5 changes: 5 additions & 0 deletions integration_tests/models/sql/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,11 @@ models:
- dbt_utils.equality:
compare_model: ref('data_star_expected')

- name: test_star_prefix_suffix
tests:
- dbt_utils.equality:
compare_model: ref('data_star_prefix_suffix_expected')

- name: test_surrogate_key
tests:
- assert_equal:
Expand Down
13 changes: 13 additions & 0 deletions integration_tests/models/sql/test_star_prefix_suffix.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{% set prefix_with = 'prefix_' if target.type != 'snowflake' else 'PREFIX_' %}
{% set suffix_with = '_suffix' if target.type != 'snowflake' else '_SUFFIX' %}

with data as (

select
{{ dbt_utils.star(from=ref('data_star'), prefix=prefix_with, suffix=suffix_with) }}

from {{ ref('data_star') }}

)

select * from data
8 changes: 4 additions & 4 deletions macros/sql/star.sql
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{% macro star(from, relation_alias=False, except=[]) -%}
{{ return(adapter.dispatch('star', 'dbt_utils')(from, relation_alias, except)) }}
{% macro star(from, relation_alias=False, except=[], prefix='', suffix='') -%}
{{ return(adapter.dispatch('star', 'dbt_utils')(from, relation_alias, except, prefix, suffix)) }}
{% endmacro %}

{% macro default__star(from, relation_alias=False, except=[]) -%}
{% macro default__star(from, relation_alias=False, except=[], prefix='', suffix='') -%}
{%- do dbt_utils._is_relation(from, 'star') -%}
{%- do dbt_utils._is_ephemeral(from, 'star') -%}

Expand All @@ -24,7 +24,7 @@

{%- for col in include_cols %}

{%- if relation_alias %}{{ relation_alias }}.{% else %}{%- endif -%}{{ adapter.quote(col)|trim }}
{%- if relation_alias %}{{ relation_alias }}.{% else %}{%- endif -%}{{ adapter.quote(col)|trim }} as {{ adapter.quote(prefix ~ col ~ suffix)|trim }}
{%- if not loop.last %},{{ '\n ' }}{% endif %}

{%- endfor -%}
Expand Down

2 comments on commit cffe8d5

@jelstongreen
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-jamie I think the automatic aliasing in the final select statement should be an optional feature of the macro - not sure about others but we don't only use the macro for sql generation but also for compiling col lists in jinja for manipulation.

We have patched this with

{% macro star(from, relation_alias=False, except=[], prefix='', suffix='', add_alias=False) -%}
    {{ return(adapter.dispatch('star', 'dbt_utils')(from, relation_alias, except, prefix, suffix)) }}
{% endmacro %}

{% macro default__star(from, relation_alias=False, except=[], prefix='', suffix='', add_alias=False) -%}
    {%- do dbt_utils._is_relation(from, 'star') -%}
    {%- do dbt_utils._is_ephemeral(from, 'star') -%}

    {#-- Prevent querying of db in parsing mode. This works because this macro does not create any new refs. #}
    {%- if not execute -%}
        {{ return('') }}
    {% endif %}

    {%- set include_cols = [] %}
    {%- set cols = adapter.get_columns_in_relation(from) -%}
    {%- set except = except | map("lower") | list %}
    {%- for col in cols -%}

        {%- if col.column|lower not in except -%}
            {% do include_cols.append(col.column) %}

        {%- endif %}
    {%- endfor %}

    {%- for col in include_cols %}

        {%- if relation_alias %}{{ relation_alias }}.{% else %}{%- endif -%}{{ adapter.quote(col)|trim }}{%- if add_alias %} as {{ adapter.quote(prefix ~ col ~ suffix)|trim }}{% else %}{%- endif -%}
        {%- if not loop.last %},{{ '\n  ' }}{% endif %}

    {%- endfor -%}
{%- endmacro %}

@joellabes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jelstongreen this is being resolved in #468 - I've just checked in to see whether the person who made the PR is available to wrap it up or not.

Please sign in to comment.