CODEWITHSUNDEEP
celeryairflowairflow-scheduler
Biplab
Biplab

Reputation: 249

where does Airflow 1.10.1 store logs for it's three services namely webserver, scheduler and worker

We recently upgraded to Airflow 1.10.1 from 1.8 and we observed that Airflow services are no more writing logs in /var/log/airflow

we are using custom commands to run airflow services and even those commands don't write to the log files specified in the commands:-

nohup airflow scheduler -sd /usr/airflow/DAGS --stdout /dev/null --stderr scheduler_error.log > /dev/null 2>nohup_error.log &

nohup airflow webserver -p 8080 --stdout /dev/null --stderr /usr/airflow/airflow_webserver_error.log --error_logfile - > /dev/null 2>/usr/airflow/nohup_airflow_webserver.log &

scheduler_error.log and airflow_webserver_error.log files remain empty.

Is the service logging mechanism broken in airflow?

Appreciate your help.

Upvotes: 2

Views: 7672

Answers (2)

y2k-shubham
y2k-shubham

Reputation: 11607

Note: this isn't really an answer, rather just an account of my limited experience with Airflow

Recently I've also been tormented by the mystery around logging in Airflow (v1.10.2). Based on my usage experience (LocalExecutor), here's what I know (please correct me if I'm wrong)

  • Webserver logs: I never found any directory / file explicitly titled "webserver_logs" (or something similar). Of course you get something on (Linux) shell when you invoke airflow webserver command. However since parsing of DAG is presently done by webserver process (checkout this thread and AIP-12), those logs can be found at base_log_folder/scheduler/date/path/to/your_file.py.log (surprise, it is in "scheduler" folder). As already pointed out by @Chirrut Imwe base_log_folder points to AIRFLOW_HOME/logs by default

  • Scheduler logs: I'm unaware of their whereabouts except for the lines I see on shell when I launch it using airflow scheduler command

  • Task logs: These are located at base_log_folder/dag_id/task_id/date_time/1.log. Of course you directly see them on the WebUI using View Log button (after clicking on you task's bubble)

  • Worker logs: Since I'm on LocalExecutor, so never came across this. But you could try to get hints from puckel/docker-airflow where they have provision for CeleryExecutor too

The UPDATING.md page has a substantial amount of stuff on logging

Upvotes: 5

Chirrut Imwe
Chirrut Imwe

Reputation: 673

have you checked $AIRFLOW_HOME/logs/ ?

Upvotes: -1

Related Questions