Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Tasks v1 readiness part3 #3202

Merged
merged 3 commits into from
Nov 25, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 47 additions & 15 deletions MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,6 @@ across different versions.

## v0.98.0 ➞ v0.99.0

### snowflake_task resource changes

new fields:
- `config`

### snowflake_tasks data source changes

New filtering options:
Expand Down Expand Up @@ -65,22 +60,40 @@ output "simple_output" {
}
```

Please adjust your Terraform configuration files.

## v0.98.0 ➞ v0.99.0

### snowflake_task resource changes
New fields:
- `config` - enables to specify JSON-formatted metadata that can be retrieved in the `sql_statement` by using [SYSTEM$GET_TASK_GRAPH_CONFIG](https://docs.snowflake.com/en/sql-reference/functions/system_get_task_graph_config).
- `show_output` and `parameters` fields added for holding SHOW and SHOW PARAMETERS output (see [raw Snowflake output](./v1-preparations/CHANGES_BEFORE_V1.md#raw-snowflake-output)).
- Added support for finalizer tasks with `finalize` field. It conflicts with `after` and `schedule` (see [finalizer tasks](https://docs.snowflake.com/en/user-guide/tasks-graphs#release-and-cleanup-of-task-graphs)).

Changes:
- `enabled` field changed to `started` and type changed to string with only boolean values available (see ["empty" values](./v1-preparations/CHANGES_BEFORE_V1.md#empty-values))
- `shedule` field changed from single value to nested object that allows for specifying either minutes or cron
- `enabled` field changed to `started` and type changed to string with only boolean values available (see ["empty" values](./v1-preparations/CHANGES_BEFORE_V1.md#empty-values)).

Before:
```terraform
resource "snowflake_task" "example" {
# ...
enabled = true
# ...
}
```
After:
```terraform
resource "snowflake_task" "example" {
# ...
started = true
# ...
}
```
- `schedule` field changed from single value to a nested object that allows for specifying either minutes or cron

Before:
```terraform
resource "snowflake_task" "example" {
# ...
schedule = "5 MINUTES"
# or
schedule = "USING SCHEDULE * * * * * UTC"
schedule = "USING CRON * * * * * UTC"
# ...
}
```
Expand All @@ -96,9 +109,28 @@ resource "snowflake_task" "example" {
# ...
}
```
- All task parameters defined in [the Snowflake documentation](https://docs.snowflake.com/en/sql-reference/parameters) added into the top-level schema and removed `session_paramters` map.
- `show_output` and `paramters` fields added for holding SHOW and SHOW PARAMETERS output (see [raw Snowflake output](./v1-preparations/CHANGES_BEFORE_V1.md#raw-snowflake-output)).
- Added support for finalizer tasks with `finalize` field. It conflicts with `after` and `schedule` (see [finalizer tasks](https://docs.snowflake.com/en/user-guide/tasks-graphs#release-and-cleanup-of-task-graphs)).
- All task parameters defined in [the Snowflake documentation](https://docs.snowflake.com/en/sql-reference/parameters) added into the top-level schema and removed `session_parameters` map.

Before:
```terraform
resource "snowflake_task" "example" {
# ...
session_parameters = {
QUERY_TAG = "<query_tag>"
}
# ...
}
```
After:
```terraform
resource "snowflake_task" "example" {
# ...
query_tag = "<query_tag>"
# ...
}
```

- `after` field type was changed from `list` to `set`. No changes in configuration are necessary.

## v0.98.0 ➞ v0.99.0

Expand Down
2 changes: 1 addition & 1 deletion docs/data-sources/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ check "task_check" {
- `limit` (Block List, Max: 1) Limits the number of rows returned. If the `limit.from` is set, then the limit wll start from the first element matched by the expression. The expression is only used to match with the first element, later on the elements are not matched by the prefix, but you can enforce a certain pattern with `starts_with` or `like`. (see [below for nested schema](#nestedblock--limit))
- `root_only` (Boolean) Filters the command output to return only root tasks (tasks with no predecessors).
- `starts_with` (String) Filters the output with **case-sensitive** characters indicating the beginning of the object name.
- `with_parameters` (Boolean) Runs SHOW PARAMETERS FOR TASK for each user returned by SHOW TASK. The output of describe is saved to the parameters field as a map. By default this value is set to true.
- `with_parameters` (Boolean) Runs SHOW PARAMETERS FOR TASK for each task returned by SHOW TASK and saves the output to the parameters field as a map. By default this value is set to true.

### Read-Only

Expand Down
150 changes: 112 additions & 38 deletions docs/resources/task.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,59 +14,133 @@ Resource used to manage task objects. For more information, check [task document
## Example Usage

```terraform
# Basic standalone task
resource "snowflake_task" "task" {
comment = "my task"

database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"

name = "task"
schedule = "10 MINUTE"
sql_statement = "select * from foo;"

session_parameters = {
"foo" : "bar",
started = true
schedule {
minutes = 5
}

user_task_timeout_ms = 10000
after = "preceding_task"
when = "foo AND bar"
enabled = true
sql_statement = "select 1"
}

# Basic serverless task
resource "snowflake_task" "serverless_task" {
comment = "my serverless task"

database = "db"
schema = "schema"

name = "serverless_task"
schedule = "10 MINUTE"
sql_statement = "select * from foo;"

session_parameters = {
"foo" : "bar",
}

user_task_timeout_ms = 10000
database = "database"
schema = "schema"
name = "task"
user_task_managed_initial_warehouse_size = "XSMALL"
after = [snowflake_task.task.name]
when = "foo AND bar"
enabled = true
started = true
schedule {
minutes = 5
}
sql_statement = "select 1"
}

resource "snowflake_task" "test_task" {
comment = "task with allow_overlapping_execution"
# Basic child task
resource "snowflake_task" "child_task" {
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
started = true
# You can do it by referring to task by computed fully_qualified_name field or write the task name in manually if it's not managed by Terraform
after = [snowflake_task.root_task.fully_qualified_name, "<database_name>.<schema_name>.<root_task_name>"]
sql_statement = "select 1"
}

database = "database"
schema = "schema"
# Basic finalizer task
resource "snowflake_task" "child_task" {
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
started = true
# You can do it by referring to task by computed fully_qualified_name field or write the task name in manually if it's not managed by Terraform
finalize = snowflake_task.root_task.fully_qualified_name
sql_statement = "select 1"
}

name = "test_task"
sql_statement = "select 1 as c;"
# Complete standalone task
resource "snowflake_task" "test" {
database = "database"
schema = "schema"
name = "task"
warehouse = "warehouse"
started = true
sql_statement = "select 1"

config = "{\"key\":\"value\"}"
allow_overlapping_execution = true
enabled = true
error_integration = "<error_integration_name>"
when = "SYSTEM$STREAM_HAS_DATA('<stream_name>')"
comment = "complete task"

schedule {
minutes = 10
}

# Session Parameters
suspend_task_after_num_failures = 10
task_auto_retry_attempts = 0
user_task_managed_initial_warehouse_size = "Medium"
user_task_minimum_trigger_interval_in_seconds = 30
user_task_timeout_ms = 3600000
abort_detached_query = false
autocommit = true
binary_input_format = "HEX"
binary_output_format = "HEX"
client_memory_limit = 1536
client_metadata_request_use_connection_ctx = false
client_prefetch_threads = 4
client_result_chunk_size = 160
client_result_column_case_insensitive = false
client_session_keep_alive = false
client_session_keep_alive_heartbeat_frequency = 3600
client_timestamp_type_mapping = "TIMESTAMP_LTZ"
date_input_format = "AUTO"
date_output_format = "YYYY-MM-DD"
enable_unload_physical_type_optimization = true
error_on_nondeterministic_merge = true
error_on_nondeterministic_update = false
geography_output_format = "GeoJSON"
geometry_output_format = "GeoJSON"
jdbc_use_session_timezone = true
json_indent = 2
lock_timeout = 43200
log_level = "OFF"
multi_statement_count = 1
noorder_sequence_as_default = true
odbc_treat_decimal_as_int = false
query_tag = ""
quoted_identifiers_ignore_case = false
rows_per_resultset = 0
s3_stage_vpce_dns_name = ""
search_path = "$current, $public"
statement_queued_timeout_in_seconds = 0
statement_timeout_in_seconds = 172800
strict_json_output = false
timestamp_day_is_always_24h = false
timestamp_input_format = "AUTO"
timestamp_ltz_output_format = ""
timestamp_ntz_output_format = "YYYY-MM-DD HH24:MI:SS.FF3"
timestamp_output_format = "YYYY-MM-DD HH24:MI:SS.FF3 TZHTZM"
timestamp_type_mapping = "TIMESTAMP_NTZ"
timestamp_tz_output_format = ""
timezone = "America/Los_Angeles"
time_input_format = "AUTO"
time_output_format = "HH24:MI:SS"
trace_level = "OFF"
transaction_abort_on_error = false
transaction_default_isolation_level = "READ COMMITTED"
two_digit_century_start = 1970
unsupported_ddl_action = "ignore"
use_cached_result = true
week_of_year_policy = 0
week_start = 0
}
```
-> **Note** Instead of using fully_qualified_name, you can reference objects managed outside Terraform by constructing a correct ID, consult [identifiers guide](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/guides/identifiers#new-computed-fully-qualified-name-field-in-resources).
Expand Down
Loading
Loading