CODEWITHSUNDEEP
pythonsystemd
jmricker
jmricker

Reputation: 19

Python script that spawns processes fails to start under systemd

I have a python script that periodically scans a folder for processing by ffmpeg. I decided to turn this into a systemd service. The script ran fine on the command-line but throws a BlockingIOError when I try to run it as a service. In an effort to figure out the issue, I boiled the script down to an almost hello world example and I'm still getting the same results. Here is what I have:

foobar.py

import subprocess 

result = subprocess.run(['/usr/bin/ffmpeg', '-h'])

print(result)

foobar.service

[Unit]
Description=foobar

[Service]
Type=simple
TasksMax=1
User=root
Environment="PATH=/usr/bin:/root/24-7"
ExecStart=/usr/bin/python3.8 /root/24-7/foobar.py

Exception

Aug 26 12:47:55 Ubuntu-1804-bionic-64-minimal systemd[1]: Started foobar.
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]: Traceback (most recent call last):
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:   File "/root/24-7/foobar.py", line 6, in <module>
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:     result = subprocess.run(['/usr/bin/ffmpeg', '-h'])
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:   File "/usr/lib/python3.8/subprocess.py", line 489, in run
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:     with Popen(*popenargs, **kwargs) as process:
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:   File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:     self._execute_child(args, executable, preexec_fn, close_fds,
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:   File "/usr/lib/python3.8/subprocess.py", line 1637, in _execute_child
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]:     self.pid = _posixsubprocess.fork_exec(
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal python3.8[4539]: BlockingIOError: [Errno 11] Resource temporarily unavailable
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal systemd[1]: foobar.service: Main process exited, code=exited, status=1/FAILURE
Aug 26 12:48:15 Ubuntu-1804-bionic-64-minimal systemd[1]: foobar.service: Failed with result 'exit-code'.

Not sure what else to try. I did I even tried delaying the call with sleep() to see if maybe waiting a few seconds would open up whatever is blocking. I'm out of ideas of what to try. Anybody have any idea?

Upvotes: 1

Views: 673

Answers (1)

alani
alani

Reputation: 13049

The problem is your TasksMax=1. This is preventing forking of child processes, and the effect is not specific to subprocess.run -- anything such as subprocess.Popen or os.system or os.fork or even non-Python applications attempting to fork would experience similar problems.

Here is os.fork displaying similar symptoms when subject to TasksMax=1:

foobar.py

import os

pid = os.fork()
if pid == 0:
    print("child")
    os._exit(0)
else:
    print(os.waitpid(pid, 0))

Aug 26 12:25:57 moon systemd[1]: Started foobar.
Aug 26 12:25:57 moon python3.8[9477]: Traceback (most recent call last):
Aug 26 12:25:57 moon python3.8[9477]:   File "/root/24-7/foobar.py", line 3, in <module>
Aug 26 12:25:57 moon python3.8[9477]:     pid = os.fork()
Aug 26 12:25:57 moon python3.8[9477]: BlockingIOError: [Errno 11] Resource temporarily unavailable
Aug 26 12:25:57 moon systemd[1]: foobar.service: Main process exited, code=exited, status=1/FAILURE
Aug 26 12:25:57 moon systemd[1]: foobar.service: Failed with result 'exit-code'.

If you remove that line or increase the limit, then it will work.

Even just increasing it to 2 works for me with your simple ffmpeg -h example, although with the real options that you want to run, you might want to increase it a little more, just in case ffmpeg needs to fork child processes of its own.

Upvotes: 2

Related Questions