0.86.0 - snowflake_external_function creation - Root resource was present, but now absent.

[flocknroll]

Terraform CLI and Provider Versions

• Running Terraform in Docker: hashicorp/terraform:1.3.9
• Snowflake-Labs/snowflake: 0.86.0

Terraform Configuration

resource "snowflake_external_function" "lambda_notifier" {
  name = "LAMBDA_NOTIFIER"
  database = var.database 
  schema   = var.schema_name
  arg {
    name = "SNS_NOTIF"
    type = "OBJECT"
  }
  return_type = "VARIANT"
  return_behavior = "VOLATILE"
  api_integration = var.devint_api_integration_name
  url_of_proxy_and_resource = "${var.devint_proxy_url_gw_sf}api/v1/trigger_json_processing"
}

Expected Behavior

The external function should be created and added to the state.

Worked with provider versions <= 0.85.0

Actual Behavior

Error at the end of the apply:

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.staging.snowflake_external_function.lambda_notifier,
│ provider "provider[\"registry.terraform.io/snowflake-labs/snowflake\"]"
│ produced an unexpected new value: Root resource was present, but now
│ absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵

The external function is actually created but not added to the state.

The next plan will mark the resource as to be created.

Steps to Reproduce

1. terraform apply

How much impact is this issue causing?

Low

Logs

No response

Additional Information

Plan output was correct:

  + resource "snowflake_external_function" "lambda_notifier" {
      + api_integration           = "API_GW_INTEGRATION"
      + comment                   = "user-defined function"
      + compression               = "AUTO"
      + created_on                = (known after apply)
      + database                  = "SNOWFLAKE_FEATURE"
      + id                        = (known after apply)
      + name                      = "LAMBDA_NOTIFIER"
      + null_input_behavior       = "CALLED ON NULL INPUT"
      + return_behavior           = "VOLATILE"
      + return_null_allowed       = true
      + return_type               = "VARIANT"
      + schema                    = "STAGING"
      + url_of_proxy_and_resource = "https://***.execute-api.eu-west-1.amazonaws.com/devint/api/v1/trigger"
      + arg {
          + name = "SNS_NOTIF"
          + type = "OBJECT"
        }
    }

[sfc-gh-asawicki]

Hey @flocknroll. Thanks for reaching out to us.

We are not able to reproduce this behavior (#2531). We tried also running a similar config locally and it was successful too. Can you provide more logs (you can run terraform with TF_LOG=DEBUG flag)?

[flocknroll]

hello @sfc-gh-asawicki,

I tried extracting relevant information from the DEBUG logs, hope this is enough:

2024-02-20T15:41:45.941+0100 [WARN]  Provider "registry.terraform.io/snowflake-labs/snowflake" produced an invalid plan for module.staging.snowflake_external_function.lambda_notifier, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .comment: planned value cty.StringVal("user-defined function") for a non-computed attribute
      - .return_null_allowed: planned value cty.True for a non-computed attribute
      - .null_input_behavior: planned value cty.StringVal("CALLED ON NULL INPUT") for a non-computed attribute
      - .compression: planned value cty.StringVal("AUTO") for a non-computed attribute

2024-02-20T15:41:46.011+0100 [DEBUG] provider.terraform-provider-snowflake_v0.86.0: 2024/02/20 15:41:46 [DEBUG] sql-conn-exec: [query CREATE EXTERNAL FUNCTION "SNOWFLAKE_FEATURE"."STAGING"."LAMBDA_NOTIFIER" (SNS_NOTIF OBJECT) RETURNS VARIANT NULL CALLED ON NULL INPUT VOLATILE COMMENT = 'user-defined function' API_INTEGRATION = "API_GW_INTEGRATION" COMPRESSION = AUTO AS 'https://***.execute-api.eu-west-1.amazonaws.com/devint/api/v1/trigger' err <nil> duration 70.689314ms args {}] ()

2024-02-20T15:41:46.671+0100 [DEBUG] provider.terraform-provider-snowflake_v0.86.0: 2024/02/20 15:41:46 [DEBUG] sql-conn-query: [query SHOW EXTERNAL FUNCTIONS LIKE 'LAMBDA_NOTIFIER' err <nil> duration 525.008526ms args {}] ()

2024-02-20T15:41:46.690+0100 [ERROR] vertex "module.staging.snowflake_external_function.lambda_notifier" error: Provider produced inconsistent result after apply

Please note that I deploy the same external function to multiple databases and multiple schemas. So the SHOW query will return multiple lines for the same function name. That may generate an inconsistency depending on how the provider handles the result.

[sfc-gh-asawicki]

Thanks @flocknroll. I will check the behavior with multiple functions named the same because this may be the source of inconsistency. I will get back to you with the results.

[sfc-gh-asawicki]

@flocknroll I have confirmed that the same name is the problem. The reason is how we handle read operations in the resource. Documentation https://docs.snowflake.com/en/sql-reference/sql/show-external-functions seems to be incorrect (no possibility to filter by db/schema, like e.g. https://docs.snowflake.com/en/sql-reference/sql/show-user-functions). It seems to be working, though (I checked manually). I have to confirm internally that this is the intended behavior, and if so, the fix will be straightforward.

[flocknroll]

Indeed,

SHOW EXTERNAL FUNCTIONS LIKE 'LAMBDA_NOTIFIER' IN SCHEMA SNOWFLAKE_FEATURE.STAGING;

seems to work fine in my context.

To be more robust, filtering on the function signature may also be a good idea. But you would have to use the information schema FUNCTIONS view or filter the current SHOW command output in the code.

[sfc-gh-asawicki]

Hey @flocknroll. We have released the fix as part of v0.87.0 release. Please follow the migration guide during the update. Please confirm that the issue is resolved in the newest version. Thanks!

[flocknroll]

hi @sfc-gh-asawicki, works fine now, thanks for your reactivity!