Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table does not exist during pipe resource creation #2075

Open
Yerachmiel-Feltzman opened this issue Sep 26, 2023 · 1 comment
Open

Table does not exist during pipe resource creation #2075

Yerachmiel-Feltzman opened this issue Sep 26, 2023 · 1 comment
Labels
bug Used to mark issues with provider's incorrect behavior category:identifiers

Comments

@Yerachmiel-Feltzman
Copy link

Provider Version

0.70.1

Terraform Version

v1.5.6

Describe the bug

When applying to create a Snowpipe the provider returns 001757 (42601): SQL compilation error: Table 'TEST_TABLE' does not exist, even though the table exists, and the database and the schema are set properly.

Expected behavior

Create the pipe in the set database and schema for the requested table.

Code samples and commands

That's what I'm doing:

resource "snowflake_pipe" "pipe" {
  name              = local.snowflake_source_bucket_pipe_name
  database          = snowflake_database.database.name
  schema            = snowflake_schema.schema.name
  auto_ingest       = true
  aws_sns_topic_arn = data.aws_sns_topic.source_bucket_sns_notification_topic.arn
  copy_statement    = <<EOF
    copy into ${snowflake_table.target_table.name}
      from @${snowflake_stage.source_bucket_stage.name}
      file_format = (format_name = ${snowflake_file_format.f_records_tsv.name})
    EOF
}

Which then returns:

│ Error: 001757 (42601): SQL compilation error:
│ Table 'TEST_TABLE' does not exist
│
│   with snowflake_pipe.pipe,
│   on snowflake.tf line 85, in resource "snowflake_pipe" "pipe":
│   85: resource "snowflake_pipe" "pipe" {
│

Additional context

Ideally, I'm using everything from other resources/data to avoid repetition, add consistency, and guarantee Terraform first creates the database, the schema, and the table, before creating the Snowpipe. For example, snowflake_table.target_table.name is from a table created in the same TF:

resource "snowflake_database" "database" {
  name = local.snowflake_database
}

resource "snowflake_schema" "schema" {
  database            = snowflake_database.database.name
  name                = var.snowflake_schema
  is_transient        = false
  is_managed          = false
  data_retention_days = var.snowflake_schema_data_retention_days
}

resource "snowflake_table" "target_table" {
  name                = var.snowflake_table
  database            = snowflake_database.database.name
  schema              = snowflake_schema.schema.name
  data_retention_days = snowflake_schema.schema.data_retention_days

  column {
    name     = "id"
    type     = "int"
    nullable = false
  }

  column {
    name     = "col_a"
    type     = "string"
    nullable = false
  }

The provider configuration:

provider "snowflake" {
  username    = ****
  private_key = ****
  account     = var.locator
  region      = var.region
  role        = "ACCOUNTADMIN"
}

As you can see, I'm running everything as ACCOUNTADMIN to avoid dealing with permissions for reporting this bug.

Available for additional clarifications.

@Yerachmiel-Feltzman Yerachmiel-Feltzman added the bug Used to mark issues with provider's incorrect behavior label Sep 26, 2023
@Yerachmiel-Feltzman Yerachmiel-Feltzman changed the title Table does not exists during pipe resource creation Table does not exist during pipe resource creation Sep 27, 2023
@chriselion
Copy link

I ran into this too. I think you need to do

...
copy into ${snowflake_table.target_table.qualified_name}
...

instead. You may also need to form the qualified name (it's not a property on the resource) for the stage too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Used to mark issues with provider's incorrect behavior category:identifiers
Projects
None yet
Development

No branches or pull requests

3 participants