CODEWITHSUNDEEP
airflowairflow-scheduler
Jaime
Jaime

Reputation: 47

How to set Airflow scheduler log file mode/permissions

I'm running airflow 1.10.3, on Red Hat Linux. I'm using a LocalExecutor, and the webserver and scheduler are both started via systemd.

The log files being generated by the scheduler are world-readable (i.e. mode "-rw-rw-rw-"). The log directories being created are "drwxrwxrwx".

This fails the security scans my organisation has in place. I need to be able to restrict the permissions on these files.

The umask in /etc/profile is 077. I've also added UMask=0007 to both the systemd unit files for the services. However, although this seems to be working for the logs in the dags/logs/scheduler/ directory, it is not affecting the DAG run logs.

[root@server logs]# ls -la s3_dag_test/
total 4
drwxrwxrwx.  4 airflow airflow   54 Aug  7 17:35 .
drwxrwx---. 46 airflow airflow 4096 Aug  7 20:00 ..
drwxrwxrwx.  5 airflow airflow  126 Aug  7 17:37 bash_test
drwxrwxrwx.  5 airflow airflow  126 Aug  7 17:29 check_s3_for_file_in_s3
[root@server logs]# ls -la s3_dag_test/bash_test/2019-08-07T17\:29\:27.988953+00\:00/
total 12
drwxrwxrwx. 2 airflow airflow   19 Aug  7 17:35 .
drwxrwxrwx. 5 airflow airflow  126 Aug  7 17:37 ..
-rw-rw-rw-. 1 airflow airflow 8241 Aug  7 17:35 1.log

Upvotes: 1

Views: 1669

Answers (2)

simlee009
simlee009

Reputation: 26

This is probably too late to be a helpful answer for you, but I had the exact same issue. My organization raised the permissions of the Airflow log directories as a security finding. I likewise checked the umask, to no avail.

I did manage to find this:

https://anoopkm.wordpress.com/2020/03/26/world-readable-airflow-dag-logs-issue/

In a nutshell, it looks like Airflow hard-codes the permissions used for creating files and folders.

I edited this Python file: venv/lib/python3.8/site-packages/airflow/utils/log/file_task_handler.py and changed lines 242 and 247 to use the 0o770 and 0o660 instead of 0o777 and 0o666 for creating folders and files, respectively. Then I manually triggered a DAG and checked the folder permissions. The newest log folder no longer had global rwx permissions.

Upvotes: 1

Ravi Ranjan
Ravi Ranjan

Reputation: 62

Can you let us know how airflow is installed as normal user or root user

Upvotes: 0

Related Questions