Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API plan/apply should allow for the pre/posthook workflows #4194

Closed
Dirrk opened this issue Jan 31, 2024 · 2 comments
Closed

API plan/apply should allow for the pre/posthook workflows #4194

Dirrk opened this issue Jan 31, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@Dirrk
Copy link

Dirrk commented Jan 31, 2024

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

When I run a plan via the api it does not run the prehook workflows (which I rely on to generate my atlantis.yaml). This basically removes the ability to use anything but the default terraform version for terragrunt using the terraform$${ATLANTIS_TERRAFORM_VERSION} value. Further, I cannot set the terraform_version in the api plan call itself which would provide a work around.

Reproduction Steps

  1. Enable the api on atlantis
  2. Curl the api to plan
curl --request POST 'http://localhost:4141/api/plan' \
  --header 'X-Atlantis-Token: REMOVED' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "Repository": "example/terraform",
    "Ref": "main",
    "Type": "Github",
    "Paths": [{
      "Directory": "aws/example",
      "Workspace": "default"
    }]
}'

Returns:
{"Error":null,"Failure":"","ProjectResults":[{"Command":1,"SubCommand":"","RepoRelDir":"aws/example","Workspace":"default","Error":{},"Failure":"","PlanSuccess":null,"PolicyCheckResults":null,"ApplySuccess":"","VersionSuccess":"","ImportSuccess":null,"StateRmSuccess":null,"ProjectName":""}],"PlansDeleted":false}

Viewing the job logs:

time=2024-01-31T18:51:33Z level=info msg=All variables passed in by terragrunt are in use. prefix=[/atlantis-data/repos/example/terraform/0/default/aws/example] 
time=2024-01-31T18:51:33Z level=info msg=All required inputs are passed in by terragrunt prefix=[/atlantis-data/repos/example/terraform/0/default/aws/example] 

Error: Unsupported Terraform Core version

  on provider.tf line 16, in terraform:
  16:   required_version = "= 1.4.6"

This configuration does not support Terraform version 0.13.5. To proceed,
either choose another supported Terraform version or update this version
constraint. Version constraints are normally set for good reason, so updating
the constraint may lead to other errors or unexpected behavior.

time=2024-01-31T18:51:33Z level=error msg=1 error occurred:
        * exit status 1

Logs

{"level":"error","ts":"2024-01-31T19:48:18.759Z","caller":"vcs/instrumented_client.go:239","msg":"Unable to update status at url: https://atlantis.example.network/jobs/333919bf-b7e1-4016-bff9-c01616ca33d6, error: POST https://api.github.com/repos/example/terraform/statuses/main: 422 Validation Failed [{Resource:Status Field:sha Code:custom Message:sha must be a 40 character SHA1}]","json":{"repository":"example/terraform","pull-num":"0","src":"atlantis/plan: aws/example/default","description":"Plan failed.","state":"failed","url":"https://atlantis.example.network/jobs/333919bf-b7e1-4016-bff9-c01616ca33d6"},"stacktrace":"github.com/runatlantis/atlantis/server/events/vcs.(*InstrumentedClient).UpdateStatus\n\t/home/runner/work/atlantis/atlantis/server/events/vcs/instrumented_client.go:239\ngithub.com/runatlantis/atlantis/server/events/vcs.(*ClientProxy).UpdateStatus\n\t/home/runner/work/atlantis/atlantis/server/events/vcs/proxy.go:84\ngithub.com/runatlantis/atlantis/server/events.(*DefaultCommitStatusUpdater).UpdateProject\n\t/home/runner/work/atlantis/atlantis/server/events/commit_status_updater.go:103\ngithub.com/runatlantis/atlantis/server/jobs.(*JobURLSetter).SetJobURLWithStatus\n\t/home/runner/work/atlantis/atlantis/server/jobs/job_url_setter.go:41\ngithub.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).updateProjectPRStatus\n\t/home/runner/work/atlantis/atlantis/server/events/project_command_runner.go:187\ngithub.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).Plan\n\t/home/runner/work/atlantis/atlantis/server/events/project_command_runner.go:164\ngithub.com/runatlantis/atlantis/server/events.RunAndEmitStats\n\t/home/runner/work/atlantis/atlantis/server/events/instrumented_project_command_runner.go:74\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).Plan\n\t/home/runner/work/atlantis/atlantis/server/events/instrumented_project_command_runner.go:38\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).apiPlan\n\t/home/runner/work/atlantis/atlantis/server/controllers/api_controller.go:152\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).Plan\n\t/home/runner/work/atlantis/atlantis/server/controllers/api_controller.go:90\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2136\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212\ngithub.com/urfave/negroni/v3.(*Negroni).UseHandler.Wrap.func1\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:59\ngithub.com/urfave/negroni/v3.HandlerFunc.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:33\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/runatlantis/atlantis/server.(*RequestLogger).ServeHTTP\n\t/home/runner/work/atlantis/atlantis/server/middleware.go:70\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Recovery).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/recovery.go:210\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Negroni).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:111\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2938\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2009"}
{"level":"error","ts":"2024-01-31T19:48:18.759Z","caller":"events/project_command_runner.go:188","msg":"updating project PR status%!(EXTRA *github.ErrorResponse=POST https://api.github.com/repos/example/terraform/statuses/main: 422 Validation Failed [{Resource:Status Field:sha Code:custom Message:sha must be a 40 character SHA1}])","json":{},"stacktrace":"github.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).updateProjectPRStatus\n\t/home/runner/work/atlantis/atlantis/server/events/project_command_runner.go:188\ngithub.com/runatlantis/atlantis/server/events.(*ProjectOutputWrapper).Plan\n\t/home/runner/work/atlantis/atlantis/server/events/project_command_runner.go:164\ngithub.com/runatlantis/atlantis/server/events.RunAndEmitStats\n\t/home/runner/work/atlantis/atlantis/server/events/instrumented_project_command_runner.go:74\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).Plan\n\t/home/runner/work/atlantis/atlantis/server/events/instrumented_project_command_runner.go:38\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).apiPlan\n\t/home/runner/work/atlantis/atlantis/server/controllers/api_controller.go:152\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).Plan\n\t/home/runner/work/atlantis/atlantis/server/controllers/api_controller.go:90\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2136\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212\ngithub.com/urfave/negroni/v3.(*Negroni).UseHandler.Wrap.func1\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:59\ngithub.com/urfave/negroni/v3.HandlerFunc.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:33\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/runatlantis/atlantis/server.(*RequestLogger).ServeHTTP\n\t/home/runner/work/atlantis/atlantis/server/middleware.go:70\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Recovery).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/recovery.go:210\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Negroni).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:111\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2938\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2009"}
{"level":"error","ts":"2024-01-31T19:48:18.759Z","caller":"events/instrumented_project_command_runner.go:78","msg":"Error running plan operation: running \"terragrunt init -input=false -upgrade --terragrunt-tfpath terraform${ATLANTIS_TERRAFORM_VERSION}\" in \"/atlantis-data/repos/example/terraform/0/default/aws/example\": exit status 1: running \"terragrunt init -input=false -upgrade --terragrunt-tfpath terraform${ATLANTIS_TERRAFORM_VERSION}\" in \"/atlantis-data/repos/example/terraform/0/default/aws/example\": \n\nError: Unsupported Terraform Core version\n\n  on provider.tf line 16, in terraform:\n  16:   required_version = \"= 1.4.6\"\n\nThis configuration does not support Terraform version 0.13.5. To proceed,\neither choose another supported Terraform version or update this version\nconstraint. Version constraints are normally set for good reason, so updating\nthe constraint may lead to other errors or unexpected behavior.\n\ntime=2024-01-31T19:48:18Z level=error msg=1 error occurred:\n\t* exit status 1\n\n\n\nIn API call\nTF Version = 0.13.5\ngenerated atlantis.yaml\n\ntime=2024-01-31T19:48:18Z level=info msg=All variables passed in by terragrunt are in use. prefix=[/atlantis-data/repos/example/terraform/0/default/aws/example] \ntime=2024-01-31T19:48:18Z level=info msg=All required inputs are passed in by terragrunt prefix=[/atlantis-data/repos/example/terraform/0/default/aws/example] \n","json":{},"stacktrace":"github.com/runatlantis/atlantis/server/events.RunAndEmitStats\n\t/home/runner/work/atlantis/atlantis/server/events/instrumented_project_command_runner.go:78\ngithub.com/runatlantis/atlantis/server/events.(*InstrumentedProjectCommandRunner).Plan\n\t/home/runner/work/atlantis/atlantis/server/events/instrumented_project_command_runner.go:38\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).apiPlan\n\t/home/runner/work/atlantis/atlantis/server/controllers/api_controller.go:152\ngithub.com/runatlantis/atlantis/server/controllers.(*APIController).Plan\n\t/home/runner/work/atlantis/atlantis/server/controllers/api_controller.go:90\nnet/http.HandlerFunc.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2136\ngithub.com/gorilla/mux.(*Router).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212\ngithub.com/urfave/negroni/v3.(*Negroni).UseHandler.Wrap.func1\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:59\ngithub.com/urfave/negroni/v3.HandlerFunc.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:33\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/runatlantis/atlantis/server.(*RequestLogger).ServeHTTP\n\t/home/runner/work/atlantis/atlantis/server/middleware.go:70\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Recovery).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/recovery.go:210\ngithub.com/urfave/negroni/v3.middleware.ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:51\ngithub.com/urfave/negroni/v3.(*Negroni).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/urfave/negroni/[email protected]/negroni.go:111\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2938\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.21.5/x64/src/net/http/server.go:2009"}

Environment details

Version 0.27.1
Helm Chart Version 4.19.0 (via Argocd onto EKS)
atlantis.yaml = empty we rely on it to be generated using terragrunt-atlantis-config during the prehook stage

server side config is mostly here:

pre_workflow_hooks:
  - run: terragrunt-atlantis-config generate --output atlantis.yaml --autoplan=true --parallel=true --ignore-parent-terragrunt --create-project-name --cascade-dependencies=false --ignore-dependency-blocks=true --automerge=true
workflows:
  default:
    plan:
      steps:
        - run: |
            if [ "$$PULL_NUM" == "0" ]; then
              terragrunt-atlantis-config generate --output atlantis.yaml --autoplan=true --parallel=true --ignore-parent-terragrunt --create-project-name --cascade-dependencies=false --ignore-dependency-blocks=true --automerge=true
            fi
        - run: terragrunt validate-inputs --terragrunt-strict-validate --terragrunt-tfpath terraform${ATLANTIS_TERRAFORM_VERSION}
        - run: terragrunt hclfmt --terragrunt-check --terragrunt-tfpath terraform${ATLANTIS_TERRAFORM_VERSION}
        - run: terragrunt init -input=false -upgrade --terragrunt-tfpath terraform${ATLANTIS_TERRAFORM_VERSION}
        - run: terragrunt plan -input=false --terragrunt-tfpath terraform${ATLANTIS_TERRAFORM_VERSION} --out $PLANFILE

Additional Context

As you can see from the above I also manually generate the atlantis.yaml as the first step of the plan but at that point the repoconfig would have already been parsed so it doesn't do it again between run commands. Ideally when I run api/plan it would work exactly as if I made a noop PR against the repo and ran atlantis plan -d path. However at minimum the api should support the terraform_version and probably any other context related variables that would be defined for a project in the atlantis.yaml

@Dirrk Dirrk added the bug Something isn't working label Jan 31, 2024
@albertorm95
Copy link
Contributor

I think this should be close now? I haven't tested this
@jamengual

@jamengual
Copy link
Contributor

closed via #4482

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants