Is there a way to check if a subprocess is still running?

Question

I'm launching a number of subprocesses with subprocess.Popen in Python. I'd like to check whether one such process has completed. I've found two ways of checking the status of a subprocess, but both seem to force the process to complete. One is using process.communicate() and printing the returncode, as explained here: checking status of process with subprocess.Popen in Python. Another is simply calling process.wait() and checking that it returns 0.

Is there a way to check if a process is still running without waiting for it to complete if it is?

Answer 1

To get this to work on multiple platforms, I use this:

    # -------------------
    def _raw_run_cmd(self, cmd):
        self._log_line(f'running "{cmd}" ...')

        shell = True
        if OsSpecific.os_name == 'win':
            cmd = ['c:/msys64/usr/bin/bash', '-i', '-c', cmd]

        proc = subprocess.Popen(cmd,
                                shell=shell,
                                bufsize=0,
                                universal_newlines=True,
                                stdin=None,
                                stdout=subprocess.PIPE,
                                stderr=subprocess.STDOUT)
        lineno = 1
        lines = []
        while True:
            line = proc.stdout.readline()
            print(f'@=@= line="{line}"')      <== for debug only
            if proc.poll() is not None:
                break
            if line:
                line = line.rstrip()
                lines.append(line)
                self._log_lineno(lineno, line)
                lineno += 1

        proc.wait()
        rc = proc.returncode
        return rc, lines

On Ubuntu 22.04 and macOS (python3.10) I was checking for an empty line ("") and that worked fine:

@=@= line="     do_ut: overall rc=0  # <= handles empty lines with \n
"
@@@@  164]      do_ut: overall rc=0
@=@= line=""                         # <= get only 1 empty line

But on Windows (running python3 in MSYS2), I was getting spurious empty lines and so the process loop was exiting too soon.

@=@= line="     do_ut: overall rc=0    # <= handles empty lines with \n
"
@@@@  164]      do_ut: overall rc=0
@=@= line=""                           # <= get multiple empty lines
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""
@=@= line=""

This shows that the proc.poll() can still return None even when there's nothing coming from the process. I thought the proc.wait() would handle that scenario but the script hung waiting for some unknown thing.

In any case, the above works for all 3 platforms.

Answer 2

If you don't mind installing comtypes (pure Python), you can use this one (tested against Python 3.10 / Windows 10):

import subprocess
from time import perf_counter

import comtypes.client


def is_process_alive(pid):
    WMI = comtypes.client.CoGetObject("winmgmts:")
    processes = WMI.InstancesOf("Win32_Process")
    if [True for pp in processes if pp.Properties_("ProcessID").Value == pid]: # checks all pids there are
        return True
    return False


p = subprocess.Popen("ping 8.8.8.8", shell=True)

start = perf_counter()
while is_process_alive(p.pid):
    print(f"{p.pid} is alive", end="\r")
print(f"{p.pid} is not alive - execution time: {perf_counter()-start}")

start = perf_counter()
p = subprocess.Popen("ping 8.8.8.8", shell=False)
while is_process_alive(p.pid):
    print(f"{p.pid} is alive", end="\r")
print(f"{p.pid} is not alive - execution time: {perf_counter()-start}")

#output
Pinging 8.8.8.8 with 32 bytes of data:
Reply from 8.8.8.8: bytes=32 time=9ms TTL=119
Reply from 8.8.8.8: bytes=32 time=11ms TTL=119
Reply from 8.8.8.8: bytes=32 time=9ms TTL=119
Reply from 8.8.8.8: bytes=32 time=10ms TTL=119
Ping statistics for 8.8.8.8:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 9ms, Maximum = 11ms, Average = 9ms
18404 is not alive - execution time: 3.170662900000025
Pinging 8.8.8.8 with 32 bytes of data:
Reply from 8.8.8.8: bytes=32 time=10ms TTL=119
Reply from 8.8.8.8: bytes=32 time=9ms TTL=119
Reply from 8.8.8.8: bytes=32 time=10ms TTL=119
Reply from 8.8.8.8: bytes=32 time=9ms TTL=119
Ping statistics for 8.8.8.8:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 9ms, Maximum = 10ms, Average = 9ms
6788 is not alive - execution time: 3.1898495000023104

Answer 3

Doing the

myProcessIsRunning = poll() is None 

As suggested by the main answer, is the recommended way and the simplest way to check if a process running. (and it works in jython as well)

If you do not have the process instance in hand to check it. Then use the operating system TaskList / Ps processes.

On windows, my command will look as follows:

filterByPid = "PID eq %s" % pid
        pidStr = str(pid)
        commandArguments = ['cmd', '/c', "tasklist", "/FI", filterByPid, "|", "findstr",  pidStr ]

This is essentially doing the same thing as the following command line:

cmd /c "tasklist /FI "PID eq 55588" | findstr 55588"

And on linux, I do exactly the same using the:

pidStr = str(pid)
commandArguments = ['ps', '-p', pidStr ]

The ps command will already be returning error code 0 / 1 depending on whether the process is found. While on windows you need the find string command.

This is the same approach that is discussed on the following stack overflow thread:

Verify if a process is running using its PID in JAVA

NOTE: If you use this approach, remember to wrap your command call in a try/except:

try:
    foundRunningProcess = subprocess.check_output(argumentsArray, **kwargs)
    return True
except Exception as err:
    return False

Note, be careful if you are developing with VS Code and using pure Python and Jython. On my environment, I was under the illusion that the poll() method did not work because a process that I suspected that must have ended was indeed running. This process had launched Wildfly. And after I had asked for wildfly to stop, the shell was still waiting for user to "Press any key to continue . . .".

In order to finish off this process, in pure python the following code was working:

process.stdin.write(os.linesep)

On jython, I had to fix this code to look as follows:

print >>process.stdin, os.linesep

And with this difference the process did indeed finish. And the jython.poll() started telling me that the process is indeed finished.

Answer 4

Ouestion: ... a way to check if a process is still running ...

You can do it for instance:

p = subprocess.Popen(...
"""
A None value indicates that the process hasn't terminated yet.
"""
poll = p.poll()
if poll is None:
  # p.subprocess is alive

Python » 3.6.1 Documentation popen-objects

Tested with Python:3.4.2

Answer 5

As suggested by the other answers None is the designed placeholder for the "return code" when no code has been returned yet by the subprocess.

The documentation for the returncode attribute backs this up (emphasis mine):

The child return code, set by poll() and wait() (and indirectly by communicate()). A None value indicates that the process hasn’t terminated yet.

A negative value -N indicates that the child was terminated by signal N (POSIX only).

An interesting place where this None value occurs is when using the timeout parameter for wait or communicate.

If the process does not terminate after timeout seconds, a TimeoutExpired exception will be raised.

If you catch that exception and check the returncode attribute it will indeed be None

import subprocess
with subprocess.Popen(['ping','127.0.0.1']) as p:
    try:
        p.wait(timeout=3)
    except subprocess.TimeoutExpired:
        assert p.returncode is None

If you look at the source for subprocess you can see the exception being raised. https://github.com/python/cpython/blob/47be7d0108b4021ede111dbd15a095c725be46b7/Lib/subprocess.py#L1930-L1931

If you search that source for self.returncode is you'll find many uses where the library authors lean on that None return code design to infer if an app is running or not running. The returncode attribute is initialized to None and only ever changes in a few spots, the main flow in invocations to _handle_exitstatus to pass on the actual return code.

Answer 6

You could use subprocess.check_output to have a look at your output.

Try this code:

import subprocess
subprocess.check_output(['your command here'], shell=True, stderr=subprocess.STDOUT)

Hope this helped!