Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add callout for net new snapshots #6521

Open
wants to merge 26 commits into
base: current
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
befa7b5
add callout for net new
mirnawong1 Nov 21, 2024
0697083
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 25, 2024
0ff5771
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 25, 2024
4127104
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 25, 2024
5a736c5
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 25, 2024
6746196
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 25, 2024
64d56e2
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 25, 2024
0347fd3
turn to snippet
mirnawong1 Nov 25, 2024
576af99
Update website/docs/reference/snapshot-configs.md
mirnawong1 Nov 25, 2024
2532cad
update grace feedback
mirnawong1 Nov 25, 2024
472d5c0
update'
mirnawong1 Nov 25, 2024
eee70d8
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 26, 2024
02fd890
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 26, 2024
1ca7c4e
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 26, 2024
43386c6
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 26, 2024
832ef12
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 26, 2024
91c3bb9
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 27, 2024
14690e0
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 27, 2024
7e68e91
update to snapshot configs
mirnawong1 Nov 27, 2024
6d2a38d
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 27, 2024
38762c1
Update website/docs/reference/snapshot-configs.md
mirnawong1 Nov 27, 2024
bdf2850
Update website/docs/reference/snapshot-configs.md
mirnawong1 Nov 27, 2024
6a84a7f
Update website/docs/reference/snapshot-configs.md
mirnawong1 Nov 27, 2024
37efd7a
Update website/snippets/_snapshots-yaml-config.md
mirnawong1 Nov 27, 2024
7eb3bf1
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 27, 2024
2fba6cc
Merge branch 'current' into net-new-snapshots
mirnawong1 Nov 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 26 additions & 9 deletions website/docs/reference/snapshot-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,18 @@

import ConfigResource from '/snippets/_config-description-resource.md';
import ConfigGeneral from '/snippets/_config-description-general.md';
import SnapshotsYamlConfig from '/snippets/_snapshots-yaml-config.md';


## Related documentation
* [Snapshots](/docs/build/snapshots)
* The `dbt snapshot` [command](/reference/commands/snapshot)

<!--
Parts of a snapshot:
- name
- query
-->

## Available configurations
### Snapshot-specific configurations


<ConfigResource meta={frontMatter.meta} />

<VersionBlock lastVersion="1.8">
Expand Down Expand Up @@ -78,8 +75,9 @@
[+](/reference/resource-configs/plus-prefix)[strategy](/reference/resource-configs/strategy): timestamp | check
[+](/reference/resource-configs/plus-prefix)[updated_at](/reference/resource-configs/updated_at): <column_name>
[+](/reference/resource-configs/plus-prefix)[check_cols](/reference/resource-configs/check_cols): [<column_name>] | all
[+](/reference/resource-configs/plus-prefix)[invalidate_hard_deletes](/reference/resource-configs/invalidate_hard_deletes) : true | false
[+](/reference/resource-configs/plus-prefix)[snapshot_meta_column_names](/reference/resource-configs/snapshot_meta_column_names): {<dictionary>}
[+](/reference/resource-configs/plus-prefix)[invalidate_hard_deletes](/reference/resource-configs/invalidate_hard_deletes) : true | false
[+](/reference/resource-configs/plus-prefix)[dbt_valid_to_current](/reference/resource-configs/dbt_valid_to_current): <string>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dbt_valid_to_current config is available in 1.9+

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh maybe I'm not reading these version blocks right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@graciegoheen You gotta expand the view so you can see line 67 / 64:

image

```

</File>
Expand Down Expand Up @@ -112,8 +110,9 @@
[strategy](/reference/resource-configs/strategy): timestamp | check
[updated_at](/reference/resource-configs/updated_at): <column_name>
[check_cols](/reference/resource-configs/check_cols): [<column_name>] | all
[snapshot_meta_column_names](/reference/resource-configs/snapshot_meta_column_names): {<dictionary>}
[invalidate_hard_deletes](/reference/resource-configs/invalidate_hard_deletes) : true | false
[snapshot_meta_column_names](/reference/resource-configs/snapshot_meta_column_names): {<dictionary>}
[dbt_valid_to_current](/reference/resource-configs/dbt_valid_to_current): <string>
```
</File>

Expand All @@ -125,7 +124,8 @@

<VersionBlock firstVersion="1.9">

Configurations can be applied to snapshots using the [YAML syntax](/docs/build/snapshots), available in Versionless and dbt v1.9 and higher, in the `snapshot` directory file.
<SnapshotsYamlConfig />


</VersionBlock>

Expand All @@ -150,11 +150,27 @@

</Tabs>

### Snapshot configuration migration

The latest snapshot YAML configurations introduced in dbt v1.9 and higher (such as [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) and [`dbt_valid_to_current`](/reference/resource-configs/dbt_valid_to_current)) are best suited for new snapshots. For existing snapshots, we recommend the following to avoid any inconsistencies to your snapshots:

Check warning on line 155 in website/docs/reference/snapshot-configs.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/reference/snapshot-configs.md#L155

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'? ", "location": {"path": "website/docs/reference/snapshot-configs.md", "range": {"start": {"line": 155, "column": 59}}}, "severity": "WARNING"}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note i'm using 'latest' so it's more scalable for the long term. if i use 'new', it'll become outdated eventually i think.

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

#### For new snapshots
- Use the latest snapshot YAML configurations when creating new snapshots that didn't exist previously.
Comment on lines +157 to +158
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just delete this?


#### For existing snapshots
- Migrate tables &mdash; Migrate the previous snapshot to the new table schema and values:
- Create a backup copy of your snapshots.
- Use `alter` statements as needed to ensure table consistency or use a script to apply the `alter` statements.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
- New YAML config &mdash; Convert the YAML configurations one at a time, testing as you go.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can just call these "new configs" we don't need to say YAML configs


If you use one of the latest configs, such as `dbt_valid_to_current`, without migrating your data may result in mixed old and new data, leading to incorrect downstream result.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved


### General configurations

<ConfigGeneral />


<Tabs
groupId="config-languages"
defaultValue="project-yaml"
Expand All @@ -170,6 +186,7 @@

<VersionBlock firstVersion="1.9">


```yaml
snapshots:
[<resource-path>](/reference/resource-configs/resource-path):
Expand Down Expand Up @@ -256,7 +273,7 @@

<VersionBlock firstVersion="1.9">

Configurations can be applied to snapshots using [YAML syntax](/docs/build/snapshots), available in Versionless and dbt v1.9 and higher, in the `snapshot` directory file.
<SnapshotsYamlConfig />

</VersionBlock>

Expand Down
1 change: 1 addition & 0 deletions website/snippets/_snapshots-yaml-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Define snapshots using the latest [YAML configurations](/docs/build/snapshots#configuring-snapshots), available in Versionless and dbt v1.9 and higher, in the `snapshot` directory file.

Check warning on line 1 in website/snippets/_snapshots-yaml-config.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/snippets/_snapshots-yaml-config.md#L1

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'? ", "location": {"path": "website/snippets/_snapshots-yaml-config.md", "range": {"start": {"line": 1, "column": 136}}}, "severity": "WARNING"}
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion website/src/components/expandable/styles.module.css
Original file line number Diff line number Diff line change
Expand Up @@ -145,4 +145,5 @@
.headerText {
display: flex;
align-items: center;
}
}

Loading