CODEWITHSUNDEEP
pythonairflow
Wessi
Wessi

Reputation: 1802

Airflow 1.10: Wrong logfile path

After upgrading to Airflow 1.10 I'm no longer able to read logs through the webserver. However Airflow is correctly producing logs and saving them to disk.

From the task instance details I can see that the listed log_filepath is wrong. According to task instance details the log_filepath is this for example: /home/ubuntu/airflow/logs/frontend_pricedata/recommendations/2018-09-11T15:10:47.296212+00:00.log But the correct filepath is /home/ubuntu/airflow/logs/frontend_pricedata/recommendations/2018-09-11T15:10:47/1.log

Is there a way to modify the log_filepath? In the config I could only find: log_filename_template = {{ ti.dag_id }}/{{ ti.task_id }}/{{ execution_date.strftime("%%Y-%%m-%%dT%%H:%%M") }}/{{ try_number }}.log which appears to be correct.

Upvotes: 2

Views: 1312

Answers (1)

fcce
fcce

Reputation: 1459

Yes, there a way to modify the log_filepath the config key is base_log_folder. But your filepath is correct , check the config file and set task_log_reader to task. In Airflow 1.9 the value is file.task.

For example:

# Log format
log_format = [%%(asctime)s] {%%(filename)s:%%(lineno)d} %%(levelname)s - %%(message)s
simple_log_format = %%(asctime)s %%(levelname)s - %%(message)s
log_filename_template = {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log
log_processor_filename_template = {{ filename }}.log
task_log_reader = task

There is a Pull Request to validate the config.

More details Airflow Writing Logs

Upvotes: 2

Related Questions