Airflow 2.4 is out, will it be supported by the latest version of the helm chart?

[andrew-sb]

By the time of this thread, Airflow's latest version is 2.4.1. I'm excited about new features and just want to confirm the compatibility of the latest helm chart with Airflow 2.4.1. I also could not find any issue or PR talking about this.

[volvocarsmatt]

I'm not an Airflow developer, but to my understanding, it is included in this PR of chart milestone 1.7.0. This one seems to suggest you can override the default version already, but that 2.4.x isn't officially supported yet.

[thesuperzapper]

I think you are confusing the user-community chart (this one), and the official chart (the one you linked).

[volvocarsmatt]

Haha, how embarrassing! I didn't even realise that there were two versions. Thanks for clarifying.

My intentions were good, but this shows you why you should not always trust advice from strangers on the internet.

[thesuperzapper]

That's all good, both charts are widely used!

[volvocarsmatt]

I started this discussion in the Airflow main page. Even though I suppose the code is there for anyone to study, I still don't get it. :(

[thesuperzapper]

@andrew-sb while I have not received enough feedback to "validate" airflow 2.4 on chart version 8.6.1, but it is highly likely that it will work fine.

If you are adventurous, you can help us ensure everything is working and report back (but please don't do it on anything critical to your business).

EDIT: I have now updated the "Airflow Version Support" table to indicate that chart version 8.6.1+ supports Airflow 2.4.

[tkansara]

Following to see the answer to the last question...

[tkansara]

I added

images:
      airflow:  
        repository: apache/airflow
        tag: 2.4.1-python3.9

to the values section of the yaml file, however the version that is installed is still 2.2.5. Any thoughts?

apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: airflow
  namespace: flux
  annotations:
    flux.weave.works/automated: "true"
spec:
  releaseName: airflow-dev1
  chart:
    repository: https://airflow-helm.github.io/charts   
    # airflow-helm.github.io/charts 
    name: airflow
    version: 8.6.1
  values:
    images:
      airflow:  
        repository: apache/airflow
        tag: 2.4.1-python3.9

    postgresql:
      enabled: true
      postgresUser: postgres
      postgresPassword: airflow
      postgresDatabase: airflow
      persistence:
        enabled: true
        size: 4Gi
        #storageClass: efs-id-1001
        existingClaim: "af-efs-fargate-1"
    fernetKey: "zTxhk7FuGqQV********************QnBnaqAItcZGI="
    env:
      AIRFLOW__KUBERNETES__DAGS_IN_IMAGE: "True"
      AIRFLOW__KUBERNETES__NAMESPACE: "dev"
      AIRFLOW__KUBERNETES__WORKER_CONTAINER_REPOSITORY: "apache/airflow"
      AIRFLOW__KUBERNETES__WORKER_CONTAINER_TAG: "1.10.10.1-alpha2-python3.7"
      AIRFLOW__KUBERNETES__RUN_AS_USER: "50000"
      AIRFLOW__CORE__LOAD_EXAMPLES: "False"
      AIRFLOW__IMAGES__AIRFLOW__TAG: "2.4.1-python3.9"
      AIRFLOW__IMAGES_AIRFLOW__REPOSITORY: "apache/airflow"
   
    executor: "KubernetesExecutor"

[thesuperzapper]

@tkansara you have specified every helm value in that example incorrectly, please base your values on one of the samples, for example:

• sample-values-CeleryExecutor.yaml
• sample-values-KubernetesExecutor.yaml
• sample-values-CeleryKubernetesExecutor.yaml

---

I imagine that in your HelmRelease case, it would look something like this:
(NOTE: ignore the specific values I have listed, they are just for reference)

apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: airflow
  namespace: flux
  annotations:
    flux.weave.works/automated: "true"
spec:
  releaseName: airflow-dev1
  chart:
    repository: https://airflow-helm.github.io/charts   
    name: airflow
    version: 8.6.1
  values:
    ########################################
    ## CONFIG | Airflow Configs
    ########################################
    airflow:
      ## if we use legacy 1.10 airflow commands
      legacyCommands: false
    
      ## configs for the airflow container image
      ## [FAQ] https://github.com/airflow-helm/charts/blob/main/charts/airflow/docs/faq/configuration/airflow-version.md
      image:
        repository: apache/airflow
        tag: 2.4.1-python3.8

      ## environment variables for airflow configs
      ## [FAQ] https://github.com/airflow-helm/charts/blob/main/charts/airflow/docs/faq/configuration/airflow-configs.md
      config:
        AIRFLOW__WEBSERVER__EXPOSE_CONFIG: "False"
        AIRFLOW__CORE__LOAD_EXAMPLES: "False"

    ###################################
    ## DATABASE | Embedded Postgres
    ###################################
    postgresql:
      ## if the `stable/postgresql` chart is used
      ## [FAQ] https://github.com/airflow-helm/charts/blob/main/charts/airflow/docs/faq/database/embedded-database.md
      ## [WARNING] the embedded Postgres is NOT SUITABLE for production deployments of Airflow
      ## [WARNING] consider using an external database with `externalDatabase.*`
      enabled: true
    
      ## configs for the PVC of postgresql
      persistence:
        enabled: true
        storageClass: ""
        size: 8Gi

[tkansara]

@thesuperzapper the values I sepcified are in my helm release, This is not the values.yaml file that is executed as helm install airflow apache-airflow/aiflow --values values.yaml

I am uploading the above file to repo, and letting flux install the helm release on the cluster. Hope that clarifies.

Regards
Tanmay

[thesuperzapper]

@tkansara that doesn't change the fact that the values you posted are not correct, for example, you are setting images.airflow.tag, when you should be setting airflow.image.tag