Ability to send environment variables for use by profiles.yml

[jennybrown8]

I'm running airflow-dbt on AWS MWAA (managed airflow). It does not let me directly configure the runtime environment of the airflow worker nodes; I can't install wrapper scripts or anything, and I don't want secrets in plain text in dag source code/yml. So I'm just calling dbt directly.

I would like to pass in secrets (mainly database username/password for connections) at runtime. The obvious way to do this is to have a profiles.yml with an env var reference, and then somehow get the DbtSeed/etc operator steps to accept an environment variables dictionary which are set just before the dbt binary is started. It would be helpful to have a clean way to do this. I have tried creating environment variables in a previous dag step, but they're already lost before the DbtOperator runs. Because these are for profiles.yml, not for the project yml, the --vars option is insufficient.

[nsoria1]

I'm would like to push this if possible. Seems like a key feature for those who are using a managed instance.

[dinigo]

I implemented this feat in the PR #45 . See the issue #39 for a working example using the MR. Feel free to contribute to the PR since there's still some things to polish (like documentation : )

[RafalSiwek]

Hi, I implemented an intermediate step with the env variable only in the PR #60, feel free to take a look.

[dinigo]

Hi, can we close this one? It is not released yet, but it is implemented in #60