Unable to get checksum for different execution environments of third party provider

[kenchan0130]

$ terraform -v
Terraform v0.14.2
$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.14.6
BuildVersion:	18G7016

Terraform Configuration Files

# sample.tf
terraform {
  required_version = "0.14.2"

  required_providers {
    datadog = {
      source  = "datadog/datadog"
      version = ">= 2.18.0"
    }
  }
}

variable "datadog_api_key" {}
variable "datadog_app_key" {}

provider "datadog" {
  api_key = var.datadog_api_key
  app_key = var.datadog_app_key
}

Debug Output

https://gist.github.com/kenchan0130/60f7f414e87866dd5c96a01d2223558c

Expected Behavior

terraform providers lock -platform=darwin_amd64 -platform=linux_amd64

I expect that this command will not cause any errors and that the lockfile will be updated.

Actual Behavior

It looks like the linux checksum cannot be verified and is causing an error.

- Fetching datadog/datadog 2.19.1 for darwin_amd64...
- Obtained datadog/datadog checksums for darwin_amd64 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)
- Fetching datadog/datadog 2.19.1 for linux_amd64...

Error: Could not retrieve providers for locking

Terraform failed to fetch the requested providers for linux_amd64 in order to
calculate their checksums: some providers could not be installed:
- registry.terraform.io/datadog/datadog: the current package for
registry.terraform.io/datadog/datadog 2.19.1 doesn't match any of the
checksums previously recorded in the dependency lock file.

Steps to Reproduce

You can reproduce this phenomenon by simply running the following command in same directory where the above file exists.

terraform init
terraform providers lock -platform=darwin_amd64 -platform=linux_amd64

References

DataDog/terraform-provider-datadog#840

[mildwonkey]

Hi @kenchan0130 ,
I am unable to reproduce this issue with the configuration you've provided:

terraform providers lock -platform=darwin_amd64 -platform=linux_amd64
- Fetching datadog/datadog 2.19.1 for darwin_amd64...
- Obtained datadog/datadog checksums for darwin_amd64 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)
- Fetching datadog/datadog 2.19.1 for linux_amd64...
- Obtained datadog/datadog checksums for linux_amd64 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)

Success! Terraform has updated the lock file.

Review the changes in .terraform.lock.hcl and then commit to your
version control system to retain the new checksums.

init and plan run fine, both before and after running the providers lock command.

Can you share what's in your .terraform.lock.hcl file? It would also help if you could share the trace logs from the command that's failing; the logs you've shared don't show any issues.

[ZymoticB]

I think I have some extended repro steps that will help here as this is also effecting me. I did all of the following with terraform v0.14.4.

It seems that if a provider already exists in my local cache (plugin_cache_dir = "$HOME/.cache/terraform") then terraform init doesn't download checksums into a fresh lock file. If the provider doesn't exist in the cache, those checksums are downloaded and you'll be unable to reproduce. I'll show that with some command sequences and the contents of .terraform.lock.hcl and hopefully that will clear up any misunderstandings I have of the internals here as I'm totally guessing.

This assumes that you have emptied your plugin cache of providers and are in a directory with just sample.tf

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding datadog/datadog versions matching ">= 2.18.0"...
- Installing datadog/datadog v2.20.0...
- Installed datadog/datadog v2.20.0 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ cat .terraform.lock.hcl
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/datadog/datadog" {
  version     = "2.20.0"
  constraints = ">= 2.18.0"
  hashes = [
    "h1:tD8lcbIsrxMP6uz58urTxfvK1LAUopyDwi5aJ5QS/LQ=",
    "zh:1c7fbaf011a39267cfbb84effeacf7105c1d39b2d7a7821687fff3b289f40cae",
    "zh:1db1daf05b27d2e537e9adff31462a3914d7ba0a1376db1eef69baa5926e0d28",
    "zh:6e1e2d10f5a3114449ef4241e153b7ed0331d120d32fc250bff8bcfd53042c4a",
    "zh:86c4a85603b32d319ad5ddf5aa3b21b406ee05ae14c37bbf444b6d09efa050aa",
    "zh:87e660d357b693b0a857edff08580cc097989c075be0c246fcb661d4969a8a25",
    "zh:8844669957186172bdc51ebefd493dd6ef08baf3a7d06ab65ad1eca4d193b007",
    "zh:951ee68c21e8de6fa5ad3ae783ab783aaaef57e69fe6c06afa24bff30e65af40",
    "zh:b98c243a0b5d3938708795ac419854a7d1b72a764442523837eef0135080d9b8",
    "zh:c4f72b76090ad146b561d0a6178ab3d8a20aea464a8035da36a4c61e4b63c2af",
    "zh:c85e16510cfa8f993942c23c20c5d79f7ff5b067791e3980adb3f76b8eed7ae6",
    "zh:e1c02bbbc1dec0b036e8843e20a6481c2ef87d9604e8d988ab7fd4abb3ee79b9",
  ]
}

This is the "good state" which @mildwonkey was unable to repro from.

Lets delete the lock and see what happens

$ rm .terraform.lock.hcl
$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding datadog/datadog versions matching ">= 2.18.0"...
- Using datadog/datadog v2.20.0 from the shared cache directory

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ cat .terraform.lock.hcl
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/datadog/datadog" {
  version     = "2.20.0"
  constraints = ">= 2.18.0"
  hashes = [
    "h1:tD8lcbIsrxMP6uz58urTxfvK1LAUopyDwi5aJ5QS/LQ=",
  ]
}

Compared to the original lock, we're missing a bunch of hashes, and in the terraform init output we see that the provider was installed from a cache.

In this state, we should be able to reliably reproduce.

$ terraform providers lock -platform=darwin_amd64 -platform=linux_amd64
- Fetching datadog/datadog 2.20.0 for darwin_amd64...

Error: Could not retrieve providers for locking

Terraform failed to fetch the requested providers for darwin_amd64 in order to
calculate their checksums: some providers could not be installed:
- registry.terraform.io/datadog/datadog: the current package for
registry.terraform.io/datadog/datadog 2.20.0 doesn't match any of the
checksums previously recorded in the dependency lock file.

As a potential workaround, it is possible to recover from this state by running terraform providers lock -platform=${the platform you have a single hash for}. For example, if you ran terraform init on linux, you can run

$ terraform providers lock -platform=linux_amd64
- Fetching datadog/datadog 2.20.0 for linux_amd64...
- Obtained datadog/datadog checksums for linux_amd64 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)

Success! Terraform has updated the lock file.

Review the changes in .terraform.lock.hcl and then commit to your
version control system to retain the new checksums.

$ terraform providers lock -platform=linux_amd64 -platform=darwin_amd64
- Fetching datadog/datadog 2.20.0 for linux_amd64...
- Obtained datadog/datadog checksums for linux_amd64 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)
- Fetching datadog/datadog 2.20.0 for darwin_amd64...
- Obtained datadog/datadog checksums for darwin_amd64 (signed by a HashiCorp partner, key ID FB70BE941301C3EA)

Success! Terraform has updated the lock file.

Review the changes in .terraform.lock.hcl and then commit to your
version control system to retain the new checksums.

$ cat .terraform.lock.hcl
# This file is maintained automatically by "terraform init".
# Manual edits may be lost in future updates.

provider "registry.terraform.io/datadog/datadog" {
  version     = "2.20.0"
  constraints = ">= 2.18.0"
  hashes = [
    "h1:LvGN54BY6A2BUW/8oy51GrMjEYiJ6neYToe239fts2o=",
    "h1:tD8lcbIsrxMP6uz58urTxfvK1LAUopyDwi5aJ5QS/LQ=",
    "zh:1c7fbaf011a39267cfbb84effeacf7105c1d39b2d7a7821687fff3b289f40cae",
    "zh:1db1daf05b27d2e537e9adff31462a3914d7ba0a1376db1eef69baa5926e0d28",
    "zh:6e1e2d10f5a3114449ef4241e153b7ed0331d120d32fc250bff8bcfd53042c4a",
    "zh:86c4a85603b32d319ad5ddf5aa3b21b406ee05ae14c37bbf444b6d09efa050aa",
    "zh:87e660d357b693b0a857edff08580cc097989c075be0c246fcb661d4969a8a25",
    "zh:8844669957186172bdc51ebefd493dd6ef08baf3a7d06ab65ad1eca4d193b007",
    "zh:951ee68c21e8de6fa5ad3ae783ab783aaaef57e69fe6c06afa24bff30e65af40",
    "zh:b98c243a0b5d3938708795ac419854a7d1b72a764442523837eef0135080d9b8",
    "zh:c4f72b76090ad146b561d0a6178ab3d8a20aea464a8035da36a4c61e4b63c2af",
    "zh:c85e16510cfa8f993942c23c20c5d79f7ff5b067791e3980adb3f76b8eed7ae6",
    "zh:e1c02bbbc1dec0b036e8843e20a6481c2ef87d9604e8d988ab7fd4abb3ee79b9",
  ]
}

I guess another workaround would be disabling your plugin cache, which I assume the issue creator has enabled.

[ZymoticB]

I see now that this has been explained in good detail over here #27264 (comment) already.

[kenchan0130]

I did see that if I eliminated the cache setting, hash was generated for multiple environments.

I close this issue as it will probably end up in #27388.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.