Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Task datasource and fixes #3195

Merged
merged 8 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,83 @@ across different versions.

> [!TIP]
> We highly recommend upgrading the versions one by one instead of bulk upgrades.

## v0.98.0 ➞ v0.99.0

### snowflake_task resource changes

Breaking changes:
- `config` field now can be defined without $ signs
sfc-gh-jmichalak marked this conversation as resolved.
Show resolved Hide resolved

Before:
```terraform
resource "snowflake_task" "config_example" {
# ...
config = "$$${\"my_config_key\":\"my_config_value\"}$$"
sfc-gh-jmichalak marked this conversation as resolved.
Show resolved Hide resolved
# ...
}
```
After:
```terraform
resource "snowflake_task" "config_example" {
# ...
config = "{\"my_config_key\":\"my_config_value\"}"
# ...
}
```

### snowflake_tasks data source changes

New filtering options:
- `with_parameters`
- `like`
- `in`
- `starts_with`
- `root_only`
- `limit`

New output fields
- `show_output`
- `parameters`

Breaking changes:
- `database` and `schema` are right now under `in` field
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved

Before:
```terraform
data "snowflake_tasks" "old_tasks" {
database = "<database_name>"
schema = "<schema_name>"
}
```
After:
```terraform
data "snowflake_tasks" "new_tasks" {
in {
# for IN SCHEMA specify:
schema = "<database_name>.<schema_name>"

# for IN DATABASE specify:
database = "<database_name>"
}
}
```
- `tasks` field now organizes output of show under `show_output` field and the output of show parameters under `parameters` field.

Before:
```terraform
output "simple_output" {
value = data.snowflake_tasks.test.tasks[0].name
}
```
After:
```terraform
output "simple_output" {
value = data.snowflake_tasks.test.tasks[0].show_output[0].name
}
```

Please adjust your Terraform configuration files.

## v0.97.0 ➞ v0.98.0

Expand Down
947 changes: 936 additions & 11 deletions docs/data-sources/tasks.md

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion docs/resources/task.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ description: |-
Resource used to manage task objects. For more information, check task documentation https://docs.snowflake.com/en/user-guide/tasks-intro.
---

!> **V1 release candidate** This resource was reworked and is a release candidate for the V1. We do not expect significant changes in it before the V1. We will welcome any feedback and adjust the resource if needed. Any errors reported will be resolved with a higher priority. We encourage checking this resource out before the V1 release. Please follow the [migration guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#v0980--v0990) to use it.

# snowflake_task (Resource)

Resource used to manage task objects. For more information, check [task documentation](https://docs.snowflake.com/en/user-guide/tasks-intro).
Expand Down Expand Up @@ -67,7 +69,6 @@ resource "snowflake_task" "test_task" {
enabled = true
}
```

-> **Note** Instead of using fully_qualified_name, you can reference objects managed outside Terraform by constructing a correct ID, consult [identifiers guide](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/guides/identifiers#new-computed-fully-qualified-name-field-in-resources).
<!-- TODO(SNOW-1634854): include an example showing both methods-->

Expand Down
110 changes: 106 additions & 4 deletions examples/data-sources/snowflake_tasks/data-source.tf
Original file line number Diff line number Diff line change
@@ -1,4 +1,106 @@
data "snowflake_tasks" "current" {
database = "MYDB"
schema = "MYSCHEMA"
}
# Simple usage
data "snowflake_tasks" "simple" {
}

output "simple_output" {
value = data.snowflake_tasks.simple.tasks
}

# Filtering (like)
data "snowflake_tasks" "like" {
like = "task-name"
}

output "like_output" {
value = data.snowflake_tasks.like.tasks
}

# Filtering (in - account - database - schema)
data "snowflake_tasks" "in_account" {
in {
account = true
}
}

data "snowflake_tasks" "in_database" {
in {
database = "<database_name>"
}
}

data "snowflake_tasks" "in_schema" {
in {
schema = "<database_name>.<schema_name>"
}
}

output "in_output" {
value = {
"account" : data.snowflake_tasks.in_account.tasks,
"database" : data.snowflake_tasks.in_database.tasks,
"schema" : data.snowflake_tasks.in_schema.tasks,
}
}

# Filtering (root only tasks)
data "snowflake_tasks" "root_only" {
root_only = true
}

output "root_only_output" {
value = data.snowflake_tasks.root_only.tasks
}

# Filtering (starts_with)
data "snowflake_tasks" "starts_with" {
starts_with = "task-"
}

output "starts_with_output" {
value = data.snowflake_tasks.starts_with.tasks
}

# Filtering (limit)
data "snowflake_tasks" "limit" {
limit {
rows = 10
from = "task-"
}
}

output "limit_output" {
value = data.snowflake_tasks.limit.tasks
}

# Without additional data (to limit the number of calls make for every found task)
data "snowflake_tasks" "only_show" {
# with_parameters is turned on by default and it calls SHOW PARAMETERS FOR task for every task found and attaches its output to tasks.*.parameters field
with_parameters = false
}

output "only_show_output" {
value = data.snowflake_tasks.only_show.tasks
}

# Ensure the number of tasks is equal to at least one element (with the use of postcondition)
data "snowflake_tasks" "assert_with_postcondition" {
starts_with = "task-name"
lifecycle {
postcondition {
condition = length(self.tasks) > 0
error_message = "there should be at least one task"
}
}
}

# Ensure the number of tasks is equal to at exactly one element (with the use of check block)
check "task_check" {
data "snowflake_tasks" "assert_with_check_block" {
like = "task-name"
}

assert {
condition = length(data.snowflake_tasks.assert_with_check_block.tasks) == 1
error_message = "tasks filtered by '${data.snowflake_tasks.assert_with_check_block.like}' returned ${length(data.snowflake_tasks.assert_with_check_block.tasks)} tasks where one was expected"
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,23 @@ package resourceparametersassert

import (
"strings"
"testing"

"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/bettertestspoc/assert"
"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk"
)

// TaskDatasourceParameters is a temporary workaround to have better parameter assertions in data source acceptance tests.
func TaskDatasourceParameters(t *testing.T, name string) *TaskResourceParametersAssert {
t.Helper()

taskAssert := TaskResourceParametersAssert{
ResourceAssert: assert.NewDatasourceAssert("data."+name, "parameters", "tasks.0."),
}
taskAssert.AddAssertion(assert.ValueSet("parameters.#", "1"))
return &taskAssert
}

func (u *TaskResourceParametersAssert) HasAllDefaults() *TaskResourceParametersAssert {
return u.
HasSuspendTaskAfterNumFailures(10).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ func (t *TaskShowOutputAssert) HasIdNotEmpty() *TaskShowOutputAssert {
return t
}

func (t *TaskShowOutputAssert) HasOwnerNotEmpty() *TaskShowOutputAssert {
t.AddAssertion(assert.ResourceShowOutputValuePresent("owner"))
return t
}

func (t *TaskShowOutputAssert) HasLastCommittedOnNotEmpty() *TaskShowOutputAssert {
t.AddAssertion(assert.ResourceShowOutputValuePresent("last_committed_on"))
return t
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,27 @@
package resourceshowoutputassert

import (
"testing"

"github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/acceptance/bettertestspoc/assert"
)

// TaskDatasourceShowOutput is a temporary workaround to have better show output assertions in data source acceptance tests.
func TaskDatasourceShowOutput(t *testing.T, name string) *TaskShowOutputAssert {
sfc-gh-jmichalak marked this conversation as resolved.
Show resolved Hide resolved
t.Helper()

taskAssert := TaskShowOutputAssert{
ResourceAssert: assert.NewDatasourceAssert("data."+name, "show_output", "tasks.0."),
}
taskAssert.AddAssertion(assert.ValueSet("show_output.#", "1"))
return &taskAssert
}

func (t *TaskShowOutputAssert) HasErrorIntegrationEmpty() *TaskShowOutputAssert {
t.AddAssertion(assert.ResourceShowOutputStringUnderlyingValueSet("error_integration", ""))
return t
}

func (u *UserShowOutputAssert) HasCreatedOnNotEmpty() *UserShowOutputAssert {
u.AddAssertion(assert.ResourceShowOutputValuePresent("created_on"))
return u
Expand Down
4 changes: 4 additions & 0 deletions pkg/acceptance/helpers/ids_generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ func (c *IdsGenerator) RandomSchemaObjectIdentifierInSchema(schemaId sdk.Databas
return sdk.NewSchemaObjectIdentifierInSchema(schemaId, c.Alpha())
}

func (c *IdsGenerator) RandomSchemaObjectIdentifierInSchemaWithPrefix(prefix string, schemaId sdk.DatabaseObjectIdentifier) sdk.SchemaObjectIdentifier {
return sdk.NewSchemaObjectIdentifierInSchema(schemaId, c.AlphaWithPrefix(prefix))
}

func (c *IdsGenerator) RandomSchemaObjectIdentifierWithArgumentsOld(arguments ...sdk.DataType) sdk.SchemaObjectIdentifier {
return sdk.NewSchemaObjectIdentifierWithArgumentsOld(c.SchemaId().DatabaseName(), c.SchemaId().Name(), c.Alpha(), arguments)
}
Expand Down
54 changes: 54 additions & 0 deletions pkg/datasources/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,37 @@ var likeSchema = &schema.Schema{
Description: "Filters the output with **case-insensitive** pattern, with support for SQL wildcard characters (`%` and `_`).",
}

var inSchema = &schema.Schema{
Type: schema.TypeList,
Optional: true,
Description: "IN clause to filter the list of objects",
MaxItems: 1,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"account": {
Type: schema.TypeBool,
Optional: true,
Description: "Returns records for the entire account.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema"},
},
"database": {
Type: schema.TypeString,
Optional: true,
Description: "Returns records for the current database in use or for a specified database.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema"},
ValidateDiagFunc: resources.IsValidIdentifier[sdk.AccountObjectIdentifier](),
},
"schema": {
Type: schema.TypeString,
Optional: true,
Description: "Returns records for the current schema in use or a specified schema. Use fully qualified name.",
ExactlyOneOf: []string{"in.0.account", "in.0.database", "in.0.schema"},
ValidateDiagFunc: resources.IsValidIdentifier[sdk.DatabaseObjectIdentifier](),
},
},
},
}

var extendedInSchema = &schema.Schema{
Type: schema.TypeList,
Optional: true,
Expand Down Expand Up @@ -114,6 +145,29 @@ func handleLimitFrom(d *schema.ResourceData, setField **sdk.LimitFrom) {
}
}

func handleIn(d *schema.ResourceData, setField **sdk.In) error {
if v, ok := d.GetOk("in"); ok {
in := v.([]any)[0].(map[string]any)
accountValue, okAccount := in["account"]
databaseValue, okDatabase := in["database"]
schemaValue, okSchema := in["schema"]

switch {
case okAccount && accountValue.(bool):
*setField = &sdk.In{Account: sdk.Bool(true)}
case okDatabase && databaseValue.(string) != "":
*setField = &sdk.In{Database: sdk.NewAccountObjectIdentifier(databaseValue.(string))}
case okSchema && schemaValue.(string) != "":
schemaId, err := sdk.ParseDatabaseObjectIdentifier(schemaValue.(string))
if err != nil {
return err
}
*setField = &sdk.In{Schema: schemaId}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (next PR): default return err?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in: #3202

}
}
return nil
}

func handleExtendedIn(d *schema.ResourceData, setField **sdk.ExtendedIn) error {
if v, ok := d.GetOk("in"); ok {
in := v.([]any)[0].(map[string]any)
Expand Down
Loading
Loading