internal/terraform: Accept only ephemeral values in ephemeral outputs

[radeksimko]

This enforces a more explicit configuration where ephemeral values have to be marked as such unless they are already ephemeral before being used in ephemeral context.

For example, the original prototype allowed this:

variable "ephemeral" {
  ephemeral = true
  default   = 12
}

locals {
  not-ephemeral = 42
}

output "foo" {
  ephemeral = true
  value     = var.ephemeral + local.not-ephemeral
}

This PR disallows that.

A new ephemeral() function may introduce an escape hatch though, to account for cases where ephemerality cannot be directly declared, such as in local values.

output "foo" {
  ephemeral = true
  value     = var.ephemeral + ephemeral(local.not-ephemeral) # ALLOWED
}

The downside is that it also enables some use cases which we probably shouldn't be enabling, for example:

output "foo" {
  ephemeral = true
  value     = ephemeral(var.not-ephemeral)
}

Ideally, variables should just be made ephemeral before referencing them. It is possible however that some users will want to reuse the same value in ephemeral and non-ephemeral context. I'm not sure how common that may be and how reasonable such a use case actually is, given the risk it introduces.

Problem

Mixing ephemeral and non-ephemeral values is more or less a problem we can solve by tracking ephemerality alongside the values.

However, the problem with turning something non-ephemeral into ephemeral is that it's difficult to tell if the value was already used in a context where it will be (or already was) persisted.

For example:

locals {
  token = # ...
}

output "foo" { # NOT PERSISTED
  ephemeral = true
  value     = ephemeral(local.token)
}

output "foo" { # PERSISTED
  value     = local.token
}

I'm not sure if we can/should be detecting such cases. I do believe however that forcing the use of the ephemeral() function will at least make people think that they're doing something unusual. This is in contrast to implying ephemerality just by reference.

Possible Limitations

Given the above context, we could restrict the use of the ephemeral() function to just local values and constants, and/or have constants imply ephemerality when mixed with ephemeral values in the same expression.

We could also implement some warning diagnostics for all the known/suspected bad use cases.

We could also hold on introducing the escape hatch initially until users come up with use cases.

I'm open to other ideas!

Tests

Tests can be added as part of #35647

[github-actions]

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.