Merge branch 'refs/heads/main' into tasks-v1-readiness

# Conflicts:
#	pkg/acceptance/bettertestspoc/README.md
#	pkg/acceptance/bettertestspoc/assert/resourceassert/gen/resource_schema_def.go
#	pkg/acceptance/helpers/notification_integration_client.go
#	pkg/internal/collections/collection_helpers.go

.github/ISSUE_TEMPLATE/01-bug.yml

@@ -43,6 +43,12 @@ body:
     validations:
       required: true
 
+  - id: company-name
+    type: input
+    attributes:
+      label: Company Name
+      description: Optional field. Providing this information allows us to communicate faster which may lead to faster issue resolution.
+
   - type: markdown
     attributes:
       value: |

.github/ISSUE_TEMPLATE/02-general-usage.yml

@@ -43,6 +43,12 @@ body:
       description: The semantic version of Terraform Provider
       placeholder: "x.y.z"
 
+  - id: company-name
+    type: input
+    attributes:
+      label: Company Name
+      description: Optional field. Providing this information allows us to communicate faster which may lead to faster issue resolution.
+
   - type: markdown
     attributes:
       value: |

.github/ISSUE_TEMPLATE/03-documentation.yml

@@ -18,6 +18,12 @@ body:
         Documentation edits are generally a bit less involved, so are often a great entrypoint if you've ever been interested in contributing!
         If you would like to contribute to the project, please let us know and refer to the [contribution guide](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/CONTRIBUTING.md) for tips on getting started.
 
+  - id: company-name
+    type: input
+    attributes:
+      label: Company Name
+      description: Optional field. Providing this information allows us to communicate faster which may lead to faster issue resolution.
+
   - id: object_type
     type: dropdown
     attributes:

.github/ISSUE_TEMPLATE/04-feature-request.yml

@@ -19,6 +19,12 @@ body:
         - [Guide on creating issues + FAQ + Commonly Known Issues](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/CREATING_ISSUES.md) - If you would like to create a GitHub issue, please read our guide first. It contains useful links, FAQ, and commonly known issues with solutions that may already solve your case.
         - [Provider Roadmap](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md)
 
+  - id: company-name
+    type: input
+    attributes:
+      label: Company Name
+      description: Optional field. Providing this information allows us to communicate faster which may lead to faster issue resolution.
+
   - id: use-case
     type: textarea
     attributes:

.github/workflows/tests.yml

@@ -41,7 +41,11 @@ jobs:
 
       - name: Create and populate .snowflake/config file
         id: create_config
-        run: mkdir $HOME/.snowflake && echo "${{ secrets.SNOWFLAKE_CONFIG_FILE }}" > $HOME/.snowflake/config
+        run: mkdir -p $HOME/.snowflake && echo "${{ secrets.SNOWFLAKE_CONFIG_FILE }}" > $HOME/.snowflake/config
+
+      - name: Create and populate .snowflake/config_v097_compatible file
+        id: create_config_v097_compatible
+        run: mkdir -p $HOME/.snowflake && echo "${{ secrets.SNOWFLAKE_CONFIG_FILE_V097_COMPATIBLE }}" > $HOME/.snowflake/config_v097_compatible
 
       - run: make test
         if: ${{ !cancelled() && steps.create_config.conclusion == 'success' }}

CHANGELOG.md

@@ -1,5 +1,52 @@
 # Changelog
 
+## [0.98.0](https://github.com/Snowflake-Labs/terraform-provider-snowflake/compare/v0.97.0...v0.98.0) (2024-11-08)
+
+
+### 🎉 **What's new:**
+
+* Add authentication policy resource ([#3098](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3098)) ([ddea819](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/ddea819cb9d0dd7918e0f4bbdaa0a2204da7b8b5))
+* Add Resource for External Volumes ([#3106](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3106)) ([64ba674](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/64ba6747f9b2364a1f84a43242472af3c4ebeca7))
+* Add stream on directory table ([#3129](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3129)) ([4391473](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/439147392436b9427e31ea578cc5bb971189a932))
+* Add stream on view ([#3150](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3150)) ([494af6d](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/494af6dd19d1098d76e7ea0451d1e144138c7a29))
+* Connection datasource ([#3173](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3173)) ([4127b3f](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/4127b3f782660772b77a72f9b38dafc728254de3))
+* Connection resource ([#3162](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3162)) ([5aef117](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/5aef117f415f238a0e786b9a063d44fadeb879e5))
+* Rework config hierarchy ([#3166](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3166)) ([04cd9f0](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/04cd9f04f1713b47f50c1931fd8955665ea8cbcc))
+* Rework provider configuration fields ([#3152](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3152)) ([fd6af43](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/fd6af43d5e50e83d247333e4a0edf85008538a9e))
+* Rework streams data source ([#3151](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3151)) ([b18bf30](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/b18bf30eee1b86e1f85c52e36122e9fa052053bd))
+* SDK Connection ([#3155](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3155)) ([bd11e0f](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/bd11e0f5eab8fe420f8af6644f4c1eb90910e69a))
+* Secret resource ([#3110](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3110)) ([16a812d](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/16a812dae20d59f31457790dcd99db03db697051))
+* Secrets datasource ([#3131](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3131)) ([8110138](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/811013887cdf1a6624c93481f18e12a183865463))
+* Upgrade tag SDK ([#3126](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3126)) ([893b288](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/893b288f91ee3d31c875f2e38c7346fe362632e6))
+
+
+### 🔧 **Misc**
+
+* Add a company name field to the issue templates ([#3182](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3182)) ([0d3248a](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/0d3248a2cb5323f94d8a984aaabe60857041fc3b))
+* Add object renaming research summary ([#3172](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3172)) ([721ee40](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/721ee40d6954254fc1f45eec50a7098b45d1afc3))
+* Add tests to 3117 and bump build time ([#3133](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3133)) ([ca90fde](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/ca90fdefc18f6c77627e602df46650e17cb54eaa))
+* Detect changes in lists and sets ([#3147](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3147)) ([c3edb79](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/c3edb79758e152a60370fcaa251fdbbdda9bcc56))
+* Exclude methods from test function checks in architest ([#3174](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3174)) ([edc46cc](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/edc46cc29fa9d0f19f5f62f04de71c5a56e4b902))
+* join object renaming tests into parameterized ones ([#3154](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3154)) ([be13502](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/be13502d7fc67c896ee1387297bddad904f515b0))
+* New roadmap entry ([#3158](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3158)) ([d83cdde](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/d83cdde59a5fb98fea5d31e5017869f841c678cb))
+* Test more authentication methods ([#3178](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3178)) ([d345cd2](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/d345cd291170db27e7ba1b83d69e613938fd9538))
+* Test support for object renaming ([#3130](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3130)) ([d665419](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/d6654195c099b81969bb8043e98771cf199ee0f4))
+
+
+### 🐛 **Bug fixes:**
+
+* Apply various fixes ([#3176](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3176)) ([55591da](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/55591da967db58a3ef11e3e454101e27ca7abb42))
+* Connection and secret-datasource tests ([#3177](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3177)) ([167de4b](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/167de4be15530d481b6e46953a7984d1b2777899))
+* Fix grant import docs ([#3183](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3183)) ([94ac910](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/94ac910f490a4cc3d9975e073e7503781bab2ffd))
+* Fix main ([#3157](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3157)) ([89b9705](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/89b9705d2af43aad3f0e098e3b8af7ced0b3d406))
+* Fix main ([#3160](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3160)) ([5b7412f](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/5b7412fe00a3620d1428d61ce30f51232eccf213))
+* Fix main ([#3186](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3186)) ([59a0a26](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/59a0a2699cbbb0a18517b607e736a04b62f6c3ba))
+* Fix user resource import ([#3181](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3181)) ([34bbbc1](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/34bbbc18b61ce1224c4f8607a12a9f6dfd95c958))
+* handle external change of secret type ([#3141](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3141)) ([649b839](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/649b8397987d13fa53c67729a81dc7fceef218a7))
+* Handle external type changes in stream resources ([#3164](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3164)) ([9fd8f88](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/9fd8f887401c0931bbbd18d3b3c4d770b8410fd4))
+* merge diffs on test clients ([#3149](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3149)) ([0f06b4a](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/0f06b4a063c9a9f9922aa4d83c47935d09757571))
+* Skip connection data source acc test ([#3184](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3184)) ([2942374](https://github.com/Snowflake-Labs/terraform-provider-snowflake/commit/2942374f668b4bdc48f81c7580262fc1c6473179))
+
 ## [0.97.0](https://github.com/Snowflake-Labs/terraform-provider-snowflake/compare/v0.96.0...v0.97.0) (2024-10-10)
 
 

MIGRATION_GUIDE.md

@@ -100,8 +100,63 @@ resource "snowflake_task" "example" {
 - `show_output` and `paramters` fields added for holding SHOW and SHOW PARAMETERS output (see [raw Snowflake output](./v1-preparations/CHANGES_BEFORE_V1.md#raw-snowflake-output)).
 - Added support for finalizer tasks with `finalize` field. It conflicts with `after` and `schedule` (see [finalizer tasks](https://docs.snowflake.com/en/user-guide/tasks-graphs#release-and-cleanup-of-task-graphs)).
 
+## v0.98.0 ➞ v0.99.0
+
+### snowflake_tag_masking_policy_association deprecation
+`snowflake_tag_masking_policy_association` is now deprecated in favor of `snowflake_tag` with a new `masking_policy` field. It will be removed with the v1 release. Please adjust your configuration files.
+
+### snowflake_tag resource changes
+New fields:
+  - `masking_policies` field that holds the associated masking policies.
+  - `show_output` field that holds the response from SHOW TAGS.
+
+#### *(breaking change)* Changed fields in snowflake_masking_policy resource
+Changed fields:
+  - `name` is now not marked as ForceNew. When this value is changed, the resource is renamed with `ALTER TAG`, instead of being recreated.
+  - `allowed_values` type was changed from list to set. This causes different ordering to be ignored.
+State will be migrated automatically.
+
+#### *(breaking change)* Identifiers related changes
+During [identifiers rework](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#identifiers-rework) we decided to
+migrate resource ids from pipe-separated to regular Snowflake identifiers (e.g. `<database_name>|<schema_name>` -> `"<database_name>"."<schema_name>"`). Importing resources also needs to be adjusted (see [example](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/tag#import)).
+
+Also, we added diff suppress function that prevents Terraform from showing differences, when only quoting is different.
+
+No change is required, the state will be migrated automatically.
+
 ## v0.97.0 ➞ v0.98.0
 
+### *(new feature)* snowflake_connections datasource
+Added a new datasource enabling querying and filtering connections. Notes:
+- all results are stored in `connections` field.
+- `like` field enables connections filtering.
+- SHOW CONNECTIONS output is enclosed in `show_output` field inside `connections`.
+  It's important to limit the records and calls to Snowflake to the minimum. That's why we recommend assessing which information you need from the data source and then providing strong filters and turning off additional fields for better plan performance.
+
+
+### *(new feature)* connection resources
+
+Added a new resources for managing connections. We decided to split connection into two separate resources based on whether the connection is a primary or replicated (secondary). i.e.:
+
+- `snowflake_primary_connection` is used to manage primary connection, with ability to enable failover to other accounts.
+- `snowflake_secondary_connection` is used to manage replicated (secondary) connection.
+
+To promote `snowflake_secondary_connection` to `snowflake_primary_connection`, resources need to be removed from the state, altered manually using:
+```
+ALTER CONNECTION <name> PRIMARY;
+```
+and then imported again, now as `snowflake_primary_connection`.
+
+To demote `snowflake_primary_connection` back to `snowflake_secondary_connection`, resources need to be removed from the state, re-created manually using:
+```
+CREATE CONNECTION <name> AS REPLICA OF <organization_name>.<account_name>.<connection_name>;
+```
+and then imported as `snowflake_secondary_connection`.
+
+For guidance on removing and importing resources into the state check [resource migration](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/docs/technical-documentation/resource_migration.md).
+
+See reference [docs](https://docs.snowflake.com/en/sql-reference/sql/create-connection).
+
 ### snowflake_streams data source changes
 New filtering options:
 - `like`
@@ -123,6 +178,31 @@ Please adjust your Terraform configuration files.
 ### *(behavior change)* Provider configuration rework
 On our road to v1, we have decided to rework configuration to address the most common issues (see a [roadmap entry](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#providers-configuration-rework)). We have created a list of topics we wanted to address before v1. We will prepare an announcement soon. The following subsections describe the things addressed in the v0.98.0.
 
+#### *(behavior change)* new fields
+We have added new fields to match the ones in [the driver](https://pkg.go.dev/github.com/snowflakedb/gosnowflake#Config) and to simplify setting account name. Specifically:
+- `include_retry_reason`, `max_retry_count`, `driver_tracing`, `tmp_directory_path` and `disable_console_login` are the new fields that are supported in the driver
+- `disable_saml_url_check` will be added to the provider after upgrading the driver
+- `account_name` and `organization_name` were added to improve handling account names. Read more in [docs](https://docs.snowflake.com/en/user-guide/admin-account-identifier#using-an-account-name-as-an-identifier).
+
+#### *(behavior change)* changed configuration of driver log level
+To be more consistent with other configuration options, we have decided to add `driver_tracing` to the configuration schema. This value can also be configured by `SNOWFLAKE_DRIVER_TRACING` environmental variable and by `drivertracing` field in the TOML file. The previous `SF_TF_GOSNOWFLAKE_LOG_LEVEL` environmental variable is not supported now, and was removed from the provider.
+
+#### *(behavior change)* deprecated fields
+Because of new fields `account_name` and `organization_name`, `account` is now deprecated. It will be removed with the v1 release. Please adjust your configurations from
+```terraform
+provider "snowflake" {
+	account = "ORGANIZATION-ACCOUNT"
+}
+```
+
+to
+```terraform
+provider "snowflake" {
+	organization_name = "ORGANIZATION"
+	account_name    = "ACCOUNT"
+}
+```
+
 #### *(behavior change)* changed behavior of some fields
 For the fields that are not deprecated, we focused on improving validations and documentation. Also, we adjusted some fields to match our [driver's](https://github.com/snowflakedb/gosnowflake) defaults. Specifically:
 - Relaxed validations for enum fields like `protocol` and `authenticator`. Now, the case on such fields is ignored.
@@ -211,6 +291,26 @@ This segregation was based on the secret flows in CREATE SECRET. i.e.:
 
 See reference [docs](https://docs.snowflake.com/en/sql-reference/sql/create-secret).
 
+### *(bugfix)* Handle BCR Bundle 2024_08 in snowflake_user resource
+
+[bcr 2024_08](https://docs.snowflake.com/en/release-notes/bcr-bundles/2024_08/bcr-1798) changed the "empty" response in the `SHOW USERS` query. This provider version adapts to the new result types; it should be used if you want to have 2024_08 Bundle enabled on your account.
+
+Note: Because [bcr 2024_07](https://docs.snowflake.com/en/release-notes/bcr-bundles/2024_07/bcr-1692) changes the way how the `default_secondary_roles` attribute behaves, drift may be reported when enabling 2024_08 Bundle. Check [Handling default secondary roles](#breaking-change-handling-default-secondary-roles) for more context.
+
+Connected issues: [#3125](https://github.com/Snowflake-Labs/terraform-provider-snowflake/issues/3125)
+
+### *(bugfix)* Handle user import correctly
+
+#### Context before the change
+
+Password is empty after the `snowflake_user` import; we can't read it from the config or from Snowflake.
+During the next terraform plan+apply it's updated to the "same" value.
+It results in an error on Snowflake side: `New password rejected by current password policy. Reason: 'PRIOR_USE'.`
+
+#### After the change
+
+The error will be ignored on the provider side (after all, it means that the password in state is the same as on Snowflake side). Still, plan+apply is needed after importing user.
+
 ## v0.96.0 ➞ v0.97.0
 
 ### *(new feature)* snowflake_stream_on_table, snowflake_stream_on_external_table resource

Makefile

@@ -74,7 +74,7 @@ test-architecture: ## check architecture constraints between packages
 	go test ./pkg/architests/... -v
 
 test-client: ## runs test that checks sdk.Client without instrumentedsql
-	SF_TF_NO_INSTRUMENTED_SQL=1 SF_TF_GOSNOWFLAKE_LOG_LEVEL=debug go test ./pkg/sdk/internal/client/... -v
+	SF_TF_NO_INSTRUMENTED_SQL=1 go test ./pkg/sdk/internal/client/... -v
 
 test-object-renaming: ## runs tests in object_renaming_acceptance_test.go
 	TEST_SF_TF_ENABLE_OBJECT_RENAMING=1 go test ./pkg/resources/object_renaming_acceptace_test.go -v