Skip to content

Commit

Permalink
wip
Browse files Browse the repository at this point in the history
  • Loading branch information
@sfc-gh-jcieslak
sfc-gh-jcieslak committed Nov 20, 2024
1 parent 6c9dc49 commit dcc89cd
Show file tree
Hide file tree
Showing 16 changed files with 1,165 additions and 178 deletions.
62 changes: 47 additions & 15 deletions MIGRATION_GUIDE.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,6 @@ across different versions.
## v0.98.0 ➞ v0.99.0

### snowflake_task resource changes

new fields:
- `config`

### snowflake_tasks data source changes

New filtering options:
Expand Down Expand Up @@ -65,22 +60,40 @@ output "simple_output" {
}
```

Please adjust your Terraform configuration files.

## v0.98.0 ➞ v0.99.0

### snowflake_task resource changes
New fields:
- `config` - enables to specify JSON-formatted metadata that can be retrieved in the `sql_statement` by using [SYSTEM$GET_TASK_GRAPH_CONFIG](https://docs.snowflake.com/en/sql-reference/functions/system_get_task_graph_config).
- `show_output` and `parameters` fields added for holding SHOW and SHOW PARAMETERS output (see [raw Snowflake output](./v1-preparations/CHANGES_BEFORE_V1.md#raw-snowflake-output)).
- Added support for finalizer tasks with `finalize` field. It conflicts with `after` and `schedule` (see [finalizer tasks](https://docs.snowflake.com/en/user-guide/tasks-graphs#release-and-cleanup-of-task-graphs)).

Changes:
- `enabled` field changed to `started` and type changed to string with only boolean values available (see ["empty" values](./v1-preparations/CHANGES_BEFORE_V1.md#empty-values))
- `shedule` field changed from single value to nested object that allows for specifying either minutes or cron
- `enabled` field changed to `started` and type changed to string with only boolean values available (see ["empty" values](./v1-preparations/CHANGES_BEFORE_V1.md#empty-values)).

Before:
```terraform
resource "snowflake_task" "example" {
# ...
enabled = true
# ...
}
```
After:
```terraform
resource "snowflake_task" "example" {
# ...
started = true
# ...
}
```
- `schedule` field changed from single value to a nested object that allows for specifying either minutes or cron

Before:
```terraform
resource "snowflake_task" "example" {
# ...
schedule = "5 MINUTES"
# or
schedule = "USING SCHEDULE * * * * * UTC"
schedule = "USING CRON * * * * * UTC"
# ...
}
```
Expand All @@ -96,9 +109,28 @@ resource "snowflake_task" "example" {
# ...
}
```
- All task parameters defined in [the Snowflake documentation](https://docs.snowflake.com/en/sql-reference/parameters) added into the top-level schema and removed `session_paramters` map.
- `show_output` and `paramters` fields added for holding SHOW and SHOW PARAMETERS output (see [raw Snowflake output](./v1-preparations/CHANGES_BEFORE_V1.md#raw-snowflake-output)).
- Added support for finalizer tasks with `finalize` field. It conflicts with `after` and `schedule` (see [finalizer tasks](https://docs.snowflake.com/en/user-guide/tasks-graphs#release-and-cleanup-of-task-graphs)).
- All task parameters defined in [the Snowflake documentation](https://docs.snowflake.com/en/sql-reference/parameters) added into the top-level schema and removed `session_parameters` map.

Before:
```terraform
resource "snowflake_task" "example" {
# ...
session_parameters = {
QUERY_TAG = "<query_tag>"
}
# ...
}
```
After:
```terraform
resource "snowflake_task" "example" {
# ...
query_tag = "<query_tag>"
# ...
}
```

- `after` field type was changed from `list` to `set`. No changes in configuration are necessary.

## v0.98.0 ➞ v0.99.0

Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/tasks.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ check "task_check" {
- `limit` (Block List, Max: 1) Limits the number of rows returned. If the `limit.from` is set, then the limit wll start from the first element matched by the expression. The expression is only used to match with the first element, later on the elements are not matched by the prefix, but you can enforce a certain pattern with `starts_with` or `like`. (see [below for nested schema](#nestedblock--limit))
- `root_only` (Boolean) Filters the command output to return only root tasks (tasks with no predecessors).
- `starts_with` (String) Filters the output with **case-sensitive** characters indicating the beginning of the object name.
- `with_parameters` (Boolean) Runs SHOW PARAMETERS FOR TASK for each user returned by SHOW TASK. The output of describe is saved to the parameters field as a map. By default this value is set to true.
- `with_parameters` (Boolean) Runs SHOW PARAMETERS FOR TASK for each task returned by SHOW TASK and saves the output to the parameters field as a map. By default this value is set to true.

### Read-Only

Expand Down
150 changes: 112 additions & 38 deletions docs/resources/task.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,59 +14,133 @@ Resource used to manage task objects. For more information, check [task document
## Example Usage

```terraform
# Basic standalone task
resource "snowflake_task" "task" {
comment = "my task"
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
name = "task"
schedule = "10 MINUTE"
sql_statement = "select * from foo;"
session_parameters = {
"foo" : "bar",
started = true
schedule {
minutes = 5
}
user_task_timeout_ms = 10000
after = "preceding_task"
when = "foo AND bar"
enabled = true
sql_statement = "select 1"
}
# Basic serverless task
resource "snowflake_task" "serverless_task" {
comment = "my serverless task"
database = "db"
schema = "schema"
name = "serverless_task"
schedule = "10 MINUTE"
sql_statement = "select * from foo;"
session_parameters = {
"foo" : "bar",
}
user_task_timeout_ms = 10000
database = "database"
schema = "schema"
name = "task"
user_task_managed_initial_warehouse_size = "XSMALL"
after = [snowflake_task.task.name]
when = "foo AND bar"
enabled = true
started = true
schedule {
minutes = 5
}
sql_statement = "select 1"
}
resource "snowflake_task" "test_task" {
comment = "task with allow_overlapping_execution"
# Basic child task
resource "snowflake_task" "child_task" {
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
started = true
# You can do it by referring to task by computed fully_qualified_name field or write the task name in manually if it's not managed by Terraform
after = [snowflake_task.root_task.fully_qualified_name, "<database_name>.<schema_name>.<root_task_name>"]
sql_statement = "select 1"
}
database = "database"
schema = "schema"
# Basic finalizer task
resource "snowflake_task" "child_task" {
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
started = true
# You can do it by referring to task by computed fully_qualified_name field or write the task name in manually if it's not managed by Terraform
finalize = snowflake_task.root_task.fully_qualified_name
sql_statement = "select 1"
}
name = "test_task"
sql_statement = "select 1 as c;"
# Complete standalone task
resource "snowflake_task" "test" {
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
started = true
sql_statement = "select 1"
config = "{\"key\":\"value\"}"
allow_overlapping_execution = true
enabled = true
error_integration = "<error_integration_name>"
when = "SYSTEM$STREAM_HAS_DATA('<stream_name>')"
comment = "complete task"
schedule {
minutes = 10
}
# Session Parameters
suspend_task_after_num_failures = 10
task_auto_retry_attempts = 0
user_task_managed_initial_warehouse_size = "Medium"
user_task_minimum_trigger_interval_in_seconds = 30
user_task_timeout_ms = 3600000
abort_detached_query = false
autocommit = true
binary_input_format = "HEX"
binary_output_format = "HEX"
client_memory_limit = 1536
client_metadata_request_use_connection_ctx = false
client_prefetch_threads = 4
client_result_chunk_size = 160
client_result_column_case_insensitive = false
client_session_keep_alive = false
client_session_keep_alive_heartbeat_frequency = 3600
client_timestamp_type_mapping = "TIMESTAMP_LTZ"
date_input_format = "AUTO"
date_output_format = "YYYY-MM-DD"
enable_unload_physical_type_optimization = true
error_on_nondeterministic_merge = true
error_on_nondeterministic_update = false
geography_output_format = "GeoJSON"
geometry_output_format = "GeoJSON"
jdbc_use_session_timezone = true
json_indent = 2
lock_timeout = 43200
log_level = "OFF"
multi_statement_count = 1
noorder_sequence_as_default = true
odbc_treat_decimal_as_int = false
query_tag = ""
quoted_identifiers_ignore_case = false
rows_per_resultset = 0
s3_stage_vpce_dns_name = ""
search_path = "$current, $public"
statement_queued_timeout_in_seconds = 0
statement_timeout_in_seconds = 172800
strict_json_output = false
timestamp_day_is_always_24h = false
timestamp_input_format = "AUTO"
timestamp_ltz_output_format = ""
timestamp_ntz_output_format = "YYYY-MM-DD HH24:MI:SS.FF3"
timestamp_output_format = "YYYY-MM-DD HH24:MI:SS.FF3 TZHTZM"
timestamp_type_mapping = "TIMESTAMP_NTZ"
timestamp_tz_output_format = ""
timezone = "America/Los_Angeles"
time_input_format = "AUTO"
time_output_format = "HH24:MI:SS"
trace_level = "OFF"
transaction_abort_on_error = false
transaction_default_isolation_level = "READ COMMITTED"
two_digit_century_start = 1970
unsupported_ddl_action = "ignore"
use_cached_result = true
week_of_year_policy = 0
week_start = 0
}
```
-> **Note** Instead of using fully_qualified_name, you can reference objects managed outside Terraform by constructing a correct ID, consult [identifiers guide](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/guides/identifiers#new-computed-fully-qualified-name-field-in-resources).
Expand Down
Loading

0 comments on commit dcc89cd

Please sign in to comment.