How to dynamically set base URL without knowing it ahead of time for load balancers?

[ketozhang]

I'm deploying multiple instances (i.e., multiple helm releases) for a multi-tenant Airflow setup. I'm exposing all of them through a common load balancer, AWS Load Balancer Controller.

The helm chart specifies if ingress.web.path is set, then config.AIRFLOW_WEBSERVER__BASE_URL must also be set. This makes sense, however I don't have ahead of time the endpoint URL as its only available during deployment .

Statically, I could register a domain name or use dynamic DNS (e.g., Elastic IPs). This proves difficult for the kind of load balancers I'm using (ALBs).

Is there any dynamic solution?

[thesuperzapper]

@ketozhang you could use some kind of templating system to generate your values, possibly based on the name of each environment, by reading an environment variable, one such system is Helmfile.

[ketozhang]

Thanks for the suggestion. I am using something similar, however the issue here is during the generation of the values file, I still do not have the URL available. It is only available after deploy of Airflow (specifically the ingress) that this is available.

In the meantime I've set the base URL to http://localhost/airflow which has side effects like the logs not using the right URL.

Another idea is using the magic of running it twice, in the second run, I can set the base URL to the correct value.