CODEWITHSUNDEEP
environment-variablesairflowairflow-scheduler
Somasundaram Sekar
Somasundaram Sekar

Reputation: 5524

Airflow configuration in environment variable not working

I tried using ENV variable to configure connection urls, I have an ami that is preconfigured with alchemy_conn and broker_url etc, I have written environment variables to /etc/environment in the instances being spun up from the amis to override the properties in airflow.cfg file. I was able to access the variables from python code as well.

But it doesn't seem to take effect while running ariflow, tried restarting the process as well but didn't work, it still points to the one in airflow.cfg file

Upvotes: 7

Views: 9127

Answers (1)

Ash Berlin-Taylor
Ash Berlin-Taylor

Reputation: 4048

The issue is probably that system services do not automatically pick up environment from /etc/environment that is just for interactive sessions.

If you are using systemd you can add EnvironmentFile=/etc/environment, though more "typical" would be to put the service specific config in /etc/default/airflow (on Debian/Ubuntu) or /etc/sysconfig/airflow (on Centos/Redhat)

The example systemd scripts from Airflow already have this for Centos: https://github.com/apache/incubator-airflow/blob/faa9a5266c0b2e68693dd106b5cb46d30770dadc/scripts/systemd/airflow-webserver.service#L20

Upvotes: 8

Related Questions