How to import packages in the DAG folder when using git-sync sidecar?

[ketozhang]

When using git-sync sidecar for DAG the repository is dumped into /opt/airflow/dags/repo by default. This is a bit problematic when you want to import some modules inside the DAG folder.

Let's say you have this repository and you want to do from foo import bar

# In /opt/airflow/dags/
repo/
  dags/
    __init__.py
    mydag.py
  foo/
    __init__.py
    bar.py

This will error due to a few things missing...

• Repository need to have __init__.py at the root level
• Because sys.path = ["/opt/airflow/dags/", ...], the import should be from repo.foo import bar

The last point is quite undesirable. When I'm in my repository it works as from foo import bar, but in Airflow it must be from repo.foo import bar.

[ketozhang]

You're going to have to add /opt/airflow/dags/repo/ to Python's sys.path. You can use the env var PYTHONPATH.

airflow:
  extraEnv:
    - name: "PYTHONPATH"
      value: "/opt/airflow/dags/repo/"