feat: Support project-level Terraform distribution selection #5167

abborg · 2024-12-15T22:13:39Z

what

Add terraform_distribution project config value
Project-level terraform_distribution config value overrides server default
Refactor terraform client to avoid cyclic dependencies

why

Enable Atlantis to manage multiple stacks or repos with different TF distributions
Supports migration from Terraform to OpenTofu project-by-project to de-risk migration

tests

I have tested my changes by running unit tests
I have tested my changes by creating a test PR that exercises the new functionality

references

Supports #3741

Add support for terraform_distribution config value in project config. This config value behaves similarly to terraform_version whereby defaults taken from the server config will be overridden by project-level values. Also refactor to prevent terraform client slightly to prevent cyclical dependencies. Signed-off-by: Andrew Borg <[email protected]>

jamengual · 2024-12-16T07:18:53Z

@abborg thanks for the contribution, please gives us some time to review.

X-Guardian · 2024-12-17T12:27:41Z

@abborg, can you update the relevant documentation in the runatlantis.io directory with your changes.

Signed-off-by: Andrew Borg <[email protected]>

abborg · 2024-12-17T14:25:55Z

I've updated the relevant documentation. Please let me know if there's any further changes desired.

Signed-off-by: Andrew Borg <[email protected]>

X-Guardian

Looking good @abborg. Just a few suggestions.

X-Guardian · 2024-12-18T13:13:27Z

cmd/server.go

+	// TFDistributionFlag is deprecated for DefaultTFDistributionFlag
+	TFDistributionFlag        = "tf-distribution"
+	TFDownloadFlag            = "tf-download"
+	TFDownloadURLFlag         = "tf-download-url"
+	UseTFPluginCache          = "use-tf-plugin-cache"
+	VarFileAllowlistFlag      = "var-file-allowlist"
+	VCSStatusName             = "vcs-status-name"
+	IgnoreVCSStatusNames      = "ignore-vcs-status-names"
+	TFEHostnameFlag           = "tfe-hostname"
+	TFELocalExecutionModeFlag = "tfe-local-execution-mode"
+	TFETokenFlag              = "tfe-token"
+	WriteGitCredsFlag         = "write-git-creds" // nolint: gosec
+	WebBasicAuthFlag          = "web-basic-auth"
+	WebUsernameFlag           = "web-username"
+	WebPasswordFlag           = "web-password"
+	WebsocketCheckOrigin      = "websocket-check-origin"


Suggested change

// TFDistributionFlag is deprecated for DefaultTFDistributionFlag

TFDistributionFlag = "tf-distribution"

TFDownloadFlag = "tf-download"

TFDownloadURLFlag = "tf-download-url"

UseTFPluginCache = "use-tf-plugin-cache"

VarFileAllowlistFlag = "var-file-allowlist"

VCSStatusName = "vcs-status-name"

IgnoreVCSStatusNames = "ignore-vcs-status-names"

TFEHostnameFlag = "tfe-hostname"

TFELocalExecutionModeFlag = "tfe-local-execution-mode"

TFETokenFlag = "tfe-token"

WriteGitCredsFlag = "write-git-creds" // nolint: gosec

WebBasicAuthFlag = "web-basic-auth"

WebUsernameFlag = "web-username"

WebPasswordFlag = "web-password"

WebsocketCheckOrigin = "websocket-check-origin"

TFDistributionFlag = "tf-distribution" // deprecated for DefaultTFDistributionFlag

TFDownloadFlag = "tf-download"

TFDownloadURLFlag = "tf-download-url"

UseTFPluginCache = "use-tf-plugin-cache"

VarFileAllowlistFlag = "var-file-allowlist"

VCSStatusName = "vcs-status-name"

IgnoreVCSStatusNames = "ignore-vcs-status-names"

TFEHostnameFlag = "tfe-hostname"

TFELocalExecutionModeFlag = "tfe-local-execution-mode"

TFETokenFlag = "tfe-token"

WriteGitCredsFlag = "write-git-creds" // nolint: gosec

WebBasicAuthFlag = "web-basic-auth"

WebUsernameFlag = "web-username"

WebPasswordFlag = "web-password"

WebsocketCheckOrigin = "websocket-check-origin"

Reduce the diff.

X-Guardian · 2024-12-18T13:20:01Z

cmd/server_test.go

@@ -73,6 +73,7 @@ var testFlags = map[string]interface{}{
 	CheckoutStrategyFlag:             CheckoutStrategyMerge,
 	CheckoutDepthFlag:                0,
 	DataDirFlag:                      "/path",
+	DefaultTFDistributionFlag:        "terraform",


Can you add a ValidateDefaultTFDistribution test?

X-Guardian · 2024-12-18T13:29:54Z

server/core/config/raw/project.go

+func validDistribution(value interface{}) error {
+	distribution := value.(*string)
+	if distribution != nil && *distribution != "terraform" && *distribution != "opentofu" {
+		return fmt.Errorf("%q is not a valid terraform_distribution, only %q and %q are supported", *distribution, "terraform", "opentofu")


Suggested change

return fmt.Errorf("%q is not a valid terraform_distribution, only %q and %q are supported", *distribution, "terraform", "opentofu")

return fmt.Errorf("'%s' is not a valid terraform_distribution, only '%s' and '%s' are supported", *distribution, "terraform", "opentofu")

%q produces a double quoted string which then needs to be escaped in the JSON log output. Single quoted strings are better as they don't have to be escaped.

X-Guardian · 2024-12-18T13:33:15Z

server/core/runtime/apply_step_runner.go

@@ -39,19 +41,27 @@ func (a *ApplyStepRunner) Run(ctx command.ProjectContext, extraArgs []string, pa

 	ctx.Log.Info("starting apply")
 	var out string
+	tfDistribution := a.DefaultTFDistribution
+	if ctx.TerraformDistribution != nil {


Is there a test that covers this condition? Same question for all runners.

X-Guardian · 2024-12-18T14:09:30Z

server/core/terraform/tfclient/terraform_client.go

@@ -14,7 +14,7 @@
 // Modified hereafter by contributors to runatlantis/atlantis.
 //
 // Package terraform handles the actual running of terraform commands.


Suggested change

// Package terraform handles the actual running of terraform commands.

// Package tfclient handles the actual running of terraform commands.

X-Guardian · 2024-12-18T14:09:53Z

server/core/terraform/tfclient/terraform_client.go

+	"github.com/runatlantis/atlantis/server/core/terraform"
+


Suggested change

"github.com/runatlantis/atlantis/server/core/terraform"

"github.com/runatlantis/atlantis/server/core/terraform"

Remove blank line

X-Guardian · 2024-12-18T14:13:14Z

server/user_config.go

+	// TFDistribution is deprecated in favor of DefaultTFDistribution
+	TFDistribution        string          `mapstructure:"tf-distribution"`


Suggested change

// TFDistribution is deprecated in favor of DefaultTFDistribution

TFDistribution string `mapstructure:"tf-distribution"`

TFDistribution string `mapstructure:"tf-distribution"` // deprecated in favor of DefaultTFDistribution

Reduce diff

X-Guardian · 2024-12-18T14:20:37Z

There are some test failures too: https://github.com/runatlantis/atlantis/actions/runs/12394308257/job/34599029347?pr=5167

abborg · 2024-12-18T15:41:12Z

There are some test failures too: https://github.com/runatlantis/atlantis/actions/runs/12394308257/job/34599029347?pr=5167

@X-Guardian I should be able to apply your suggestions and fix remaining test failures later today.

abborg · 2024-12-20T13:46:03Z

@X-Guardian I made a mistake when attempting to correct a commit missing a signoff and it appears I mistakenly signed off some commits from the earlier merges of main. Please let me know how you'd like me to proceed.

Edit: I was able to checkout the previous head, make the correct signoff and then push the changes.

Signed-off-by: Andrew Borg <[email protected]>

obscurerichard · 2024-12-21T15:58:32Z

It looks like this relates to the OpenTofu epic #3741 - does this also provide a resolution to Support OpenTofu auto download #4339?

(I'm not one of the Atlantis developers, just an interested Atlantis user starting to experiment with OpenTofu - and this is very exciting to see!)

abborg requested review from a team as code owners December 15, 2024 22:13

abborg requested review from chenrui333, lukemassa and nitrocode and removed request for a team December 15, 2024 22:13

github-actions bot added the go Pull requests that update Go code label Dec 15, 2024

dosubot bot added the feature New functionality/enhancement label Dec 15, 2024

abborg changed the title ~~Support project-level Terraform distribution selection~~ feat:Support project-level Terraform distribution selection Dec 15, 2024

abborg changed the title ~~feat:Support project-level Terraform distribution selection~~ feat: Support project-level Terraform distribution selection Dec 15, 2024

abborg force-pushed the main branch from 6718320 to 20e0cc3 Compare December 15, 2024 22:37

abborg and others added 2 commits December 15, 2024 17:41

Merge branch 'main' into main

4f9dd08

Merge branch 'main' into main

ac3a7f6

X-Guardian added the waiting-on-response Waiting for a response from the user label Dec 17, 2024

Update documentation for Terraform distribution

fc81e67

Signed-off-by: Andrew Borg <[email protected]>

github-actions bot added the docs Documentation label Dec 17, 2024

abborg and others added 2 commits December 17, 2024 13:14

Fix mis-named deprecation warning

ee74bb7

Signed-off-by: Andrew Borg <[email protected]>

Merge branch 'main' into main

13ab59e

X-Guardian requested changes Dec 18, 2024

View reviewed changes

abborg force-pushed the main branch from 4aed6c9 to 49d2808 Compare December 20, 2024 13:39

github-actions bot added build Relating to how we build Atlantis github-actions labels Dec 20, 2024

abborg added 2 commits December 20, 2024 09:11

Fix markdown formatting

f9833be

Signed-off-by: Andrew Borg <[email protected]>

Add tests for TF Distribution and fix styles

e7cadaf

Signed-off-by: Andrew Borg <[email protected]>

abborg force-pushed the main branch from 49d2808 to e7cadaf Compare December 20, 2024 15:50

Merge branch 'main' into main

ad07523

This was referenced Dec 21, 2024

Support OpenTofu (epic) #3741

Open

Download multiple versions of OpenTofu in container by default, including 1.6.2, 1.7.1, and 1.8.2 #5187

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Support project-level Terraform distribution selection #5167

feat: Support project-level Terraform distribution selection #5167

abborg commented Dec 15, 2024

jamengual commented Dec 16, 2024

X-Guardian commented Dec 17, 2024

abborg commented Dec 17, 2024

X-Guardian left a comment

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian Dec 18, 2024

X-Guardian commented Dec 18, 2024

abborg commented Dec 18, 2024

abborg commented Dec 20, 2024 •

edited

Loading

obscurerichard commented Dec 21, 2024 •

edited

Loading

	return fmt.Errorf("%q is not a valid terraform_distribution, only %q and %q are supported", *distribution, "terraform", "opentofu")
	return fmt.Errorf("'%s' is not a valid terraform_distribution, only '%s' and '%s' are supported", *distribution, "terraform", "opentofu")

	// Package terraform handles the actual running of terraform commands.
	// Package tfclient handles the actual running of terraform commands.

	"github.com/runatlantis/atlantis/server/core/terraform"
	"github.com/runatlantis/atlantis/server/core/terraform"

		// TFDistribution is deprecated in favor of DefaultTFDistribution
		TFDistribution string `mapstructure:"tf-distribution"`

feat: Support project-level Terraform distribution selection #5167

Are you sure you want to change the base?

feat: Support project-level Terraform distribution selection #5167

Conversation

abborg commented Dec 15, 2024

what

why

tests

references

jamengual commented Dec 16, 2024

X-Guardian commented Dec 17, 2024

abborg commented Dec 17, 2024

X-Guardian left a comment

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian Dec 18, 2024

Choose a reason for hiding this comment

X-Guardian commented Dec 18, 2024

abborg commented Dec 18, 2024

abborg commented Dec 20, 2024 • edited Loading

obscurerichard commented Dec 21, 2024 • edited Loading

abborg commented Dec 20, 2024 •

edited

Loading

obscurerichard commented Dec 21, 2024 •

edited

Loading