CODEWITHSUNDEEP
airflow
user3368526
user3368526

Reputation: 2328

How do you stop/check logs for airflow webserver/scheduler with --daemon?

Hi I'm using Airflow and running in background in EC2. But how do you stop/check logs? I couldn't find the detailed explanation on the documentation.

Thanks!

Upvotes: 7

Views: 24085

Answers (3)

dorvak
dorvak

Reputation: 9709

If you start your Airflow-Webserver and other deamon using systemd (see this repo), it`s quite straightforward:

journalctl -u airflow-webserver.service

Upvotes: 5

apeletz
apeletz

Reputation: 81

I was able to find the Airflow webserver logs under /var/log/upstart/ even though the designated log location was set to /var/log/airflow/. I originally thought the webserver process was not producing any logs but turns out it was just in a different location than specified in the config files.

Upvotes: 6

saarp
saarp

Reputation: 1951

Job/task logs are available via the admin UI. I have given up on webserver logs. I have never seen any output logging when passing logging parameters (-l, --stdout, --stderr) to the airflow webserver command.

I do capture scheduler output by launching it as follows:

nohup airflow scheduler >> ${AIRFLOW_HOME}/logs/scheduler.log 2>&1 &

The Airflow webserver will create a PID file in the AIRFLOW_HOME folder so you stop it with kill -TERM $(cat ${AIRFLOW_HOME}/airflow-webserver.pid or use pkill -f "gunicorn: master [airflow-webserver]".

Upvotes: 7

Related Questions