Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot config to define column names #6146

Closed
wants to merge 5 commits into from
Closed

Snapshot config to define column names #6146

Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
22e3302
Snapshot config to define column names
nataliefiann Sep 25, 2024
906cbdb
Update website/docs/reference/snapshot-configs.md
nataliefiann Sep 25, 2024
2995efb
Update website/docs/reference/snapshot-configs.md
nataliefiann Sep 25, 2024
d7f515c
Update website/docs/reference/snapshot-configs.md
nataliefiann Sep 30, 2024
2093a89
Update website/docs/reference/snapshot-configs.md
nataliefiann Sep 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions website/docs/reference/snapshot-configs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,3 +301,48 @@ snapshots:
```

</File>

<VersionBlock firstVersion="1.9">

#### Define column names in snapshot table
nataliefiann marked this conversation as resolved.
Show resolved Hide resolved

Use the following configuration to define the names of the columns created in the [snapshot table](/docs/build/snapshots#snapshot-meta-fields).
nataliefiann marked this conversation as resolved.
Show resolved Hide resolved

<File name='dbt_project.yml'>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's also show the example from the issue on how to do this in the config block:

{{
    config(
      target_database='analytics',
      target_schema='snapshots',
      unique_key='id',

      strategy='timestamp',
      updated_at='updated_at',
      
      snapshot_meta_column_names={
        dbt_valid_from: start_date,
        dbt_valid_to: end_date,
        dbt_scd_id: scd_id,
        dbt_updated_at: updated_date
      }
    )
}}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have some tab thing for other examples so you can see how to write a config in different places


```yml

snapshots:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use the full example from the issue!

snapshots:
  my_project:
    +snapshot_meta_column_names={
      dbt_valid_from: start_date,
      dbt_valid_to: end_date,
      dbt_scd_id: scd_id,
      dbt_updated_at: updated_date
    }

my_project:
+snapshot_meta_column_names ={
dbt_valid_from: example_valid_from,
dbt_valid_to: example_valid_to
}

```

</File>

- When you run your snapshots, dbt handles the renames from `dbt_valid_from` to `example_valid_from` for you.
- If you make subsequent updates to the config to rename the column, dbt manages the migration of column names for each configuration change. For example:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not true.

See issue -> #6112 (comment)

we aren't doing anything for migration, so if a user renames the columns for a snapshot without updating an existing table, they will get an error. The recommendation is to use these settings for new snapshots, or to arrange to update their existing tables prior to committing a column name change.


<File name='dbt_project.yml'>

```yml

snapshots:
my_project:
+snapshot_meta_column_names ={
dbt_valid_from: test_valid_from,
dbt_valid_to: test_valid_to
}

```

</File>

- When you run your snapshots, dbt handles the renames from `example_valid_from` to `test_valid_from` for you.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment as above - this is not true


You can read through the section on [Defining column names](/docs/build/snapshots#define-column-names-in-snapshot-table) for more information.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the link doesn't take me to the right place? or for some reason i didn't see the 'define column names' section?


</VersionBlock>
Loading