Provider cache server was unable to process the received request, code=400

[mwos-sl]

Describe the bug

When running terragrunt 0.69.0 and some older terraform version (I saw this for tf 0.13.5, 0.14.3, 1.1.2, 1.1.9) I'm getting the error on terragrunt plan:

21:17:41.807 INFO   Terragrunt Cache server is listening on 127.0.0.1:36757
21:17:41.809 INFO   Start Terragrunt Cache server
21:18:21.400 INFO   Downloading Terraform configurations from git::ssh://[email protected]/<anonymised> into ./.terragrunt-cache/gK-KBwohpR10ufYxkwT9yO68-HI/Oc7GmR7V1D-J0aE1XreS8HCGi24
21:18:23.319 INFO   Caching terraform providers for ./.terragrunt-cache/gK-KBwohpR10ufYxkwT9yO68-HI/Oc7GmR7V1D-J0aE1XreS8HCGi24/modules/base
21:18:24.638 ERROR  Cache server was unable to process the received request, code=400, message=missing key in request header

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider hashicorp/aws:
could not query provider registry for registry.terraform.io/hashicorp/aws: 400
Bad Request

21:18:24.647 ERROR  terraform invocation failed in ./.terragrunt-cache/gK-KBwohpR10ufYxkwT9yO68-HI/Oc7GmR7V1D-J0aE1XreS8HCGi24/modules/base
21:18:24.647 INFO   Shutting down Terragrunt Cache server...
21:18:24.647 INFO   Terragrunt Cache server stopped
21:18:24.647 ERROR  error occurred:

* Failed to execute "terraform init -backend-config=region=us-west-2 -backend-config=bucket=<anonymised> -backend-config=dynamodb_table=infrastructure-live-infrastructure -backend-config=encrypt=true -backend-config=key=<anonymised>/global/terraform.tfstate -no-color -no-color" in ./.terragrunt-cache/gK-KBwohpR10ufYxkwT9yO68-HI/Oc7GmR7V1D-J0aE1XreS8HCGi24/modules/base
  
  Error: Failed to query available provider packages
  
  Could not retrieve the list of available versions for provider hashicorp/aws:
  could not query provider registry for registry.terraform.io/hashicorp/aws: 400
  Bad Request
  
  
  exit status 1

The problem is NOT there when using terraform versions: 1.2.0, 1.2.9, 1.3.0, 1.3.10, 1.5.7.

I haven't tested other versions of terraform, but based on these, it seems that breaking point is between 1.1.9 and 1.2.0 terraform releases, so maybe something along the lines: https://github.com/hashicorp/terraform/blob/v1.2/CHANGELOG.md#120-may-18-2022

Steps To Reproduce

Steps to reproduce the behavior:

1. Enable terragrunt provider cache.
2. run terragrunt plan on whatever project with terraform 1.1.2 (other terraform might be also affected - the list of OK and not OK that I found so far is in the previous point)

Expected behavior

Terragrunt does not fail when running some older terraform.

Versions

• Terragrunt version: 0.69.0
• Terraform version: <= 1.1.9
• Environment details: Ubuntu 24.04

Additional context

I'm getting this error both from laptop and from container running atlantis.

[mwos-sl]

FYI @levkohimins

[mwos-sl]

EDIT: Updated original message with tf version breaking point.