snowflake_stage recreates every apply regardless of an actual change

[bmhah]

Terraform CLI and Provider Versions

v1.5.0
0.85.0

Terraform Configuration

resource "snowflake_stage" "LANDING_8X8_STAGE_INBOUND_CALLS_S3" {
  name                = "LANDING_8X8_STAGE_INBOUND_CALLS"
  url                 = var.s3_uri_8x8_inbound_calls
  database            = "AWS_LANDING_INGEST_DB_${var.SF_ENVIRONMENT}"
  schema              = "LANDING_8X8_SCHEMA"
  comment             = "Landing stage for inbound calls s3 bucket"
  file_format         = "FORMAT_NAME = AWS_LANDING_INGEST_DB_${var.SF_ENVIRONMENT}.LANDING_8X8_SCHEMA.LANDING_FF_CSV"
  copy_options        = "ON_ERROR = 'CONTINUE'"
  storage_integration = var.aws_storage_integration
}

Expected Behavior

Stage doesn't destroy and recreate

Actual Behavior

Stage recreates, this causes snowflake pipes to break having a status STOPPED_STAGE_DROPPED

Steps to Reproduce

1. terraform apply

How much impact is this issue causing?

High

Logs

No response

Additional Information

No response

[sfc-gh-asawicki]

Hey @bmhah. Thanks for reaching out to us.

Please share the terraform run logs with TF_LOG=DEBUG enabled.

[sfc-gh-bhill]

Here is the file that the customer provided me. plan.txt.gz

[bmhah]

Any updates here?

[sfc-gh-asawicki]

Hey @bmhah. I will confirm in the morning, but it looks like an easy fix, so we may even be able to include it in tomorrow's release. I will keep you posted.

[sfc-gh-asawicki]

Hey @bmhah. We have released the new version: https://github.com/Snowflake-Labs/terraform-provider-snowflake/releases/tag/v0.89.0. Please bump and check if the error persists. Please remember to use the migration guide during version upgrades: https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/MIGRATION_GUIDE.md#migration-guide.

[bmhah]

Sounds good, we'll get to testing this later this week. I'll keep this issue open in the meantime

[sfc-gh-asawicki]

Hey @bmhah. Have you have a chance to test the newer versions after the introduced fixes?

[bmhah]

Yes, we've done the provider upgrade to 88 and have also used others, they haven't worked for us, the stage for this pipe still drops and recreates causing the pipe to break again

[sfc-gh-asawicki]

The fix was introduced in 0.89, not 0.88, so please provide the config and logs (run terraform with TF_LOG=DEBUG) on the 0.89+ version (I could not reproduce the issue given your config on the 0.89 version).

[guillaumelecerf]

We were having the same issue with:

file_format         = "FIELD_DELIMITER = '|' PARSE_HEADER = true"

Fix in #2885

[sfc-gh-asawicki]

The fix was included in v0.93.0. Please check it @guillaumelecerf @bmhah.

[harukitake]

Hey @sfc-gh-asawicki.
I have the same issue on the version 0.93.0.

terraform configuration

resource "snowflake_stage" "stage" {
      database = "DATABASE_NAME"
      file_format = "TYPE = CSV NULL_IF = []"
      name = "NAME"
      schema = "SCHEMA_NAME"
      storage_integration = "INTEGRATION_NAME"
      url = "s3://aaaaa"
    }

[sfc-gh-asawicki]

I will take a look this week. Thanks for providing the config; it helps reproduce the issues.

[harukitake]

Hi @sfc-gh-asawicki.
Do you have any updates for this issue?

[sfc-gh-asawicki]

Hey @harukitake. Sorry, I missed this one, I will check it this week.

[sfc-gh-asawicki]

Hey @harukitake. I've finally reproduced the error.

The current implementation of the stage resource filters out all the file format properties that have default values in Snowflake. TYPE = CSV is the default value, so after reading the resource's state, it is programmatically removed from the file format string, which causes the permadiff you see.

This is, of course, not the expected behavior. For now, please just remove TYPE = CSV from your config, because this is the default in Snowflake.

We will fix the handling of these parameters during the stage resource rework in the next few weeks. I have added a test to confirm it and the note in the resource documentation.