CODEWITHSUNDEEP
pythonsubprocess
nao
nao

Reputation: 1188

Non-blocking version of Popen.communicate() in Python Subprocess module

I'm trying to execute multiple commands in parallel in two shells spawned in a Python program. I've tried to use popen.communicate(), however this method is blocking and does not allow me to run them simultaneously. I currently have.

proc_1 = subprocess.Popen('adb shell', shell=True, stdin=subprocess.PIPE)
proc_2 = subprocess.Popen('adb shell', shell=True, stdin=subprocess.PIPE)
proc_1.communicate(b'second adb command')
proc_2.communicate(b'second adb command')

It's currently waiting for the first communicate call to finish before proceeding to the second. How could I run them both in parallel?

Upvotes: 0

Views: 997

Answers (1)

azelcer
azelcer

Reputation: 1533

This is the expected behavior of communicate

Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate and set the returncode attribute.

Since you do not need to manage stdout nor stderr, it should be enough to call communicate with a timeout of 0 and catch the TimeoutExpired exception:

import subprocess

def communicate_nowait(proc, message):
    try:
        proc.communicate(message, 1E-3)  # 0 might have problems
    except subprocess.TimeoutExpired:
        pass
    
proc_1 = subprocess.Popen('adb shell', shell=True, stdin=subprocess.PIPE)
proc_2 = subprocess.Popen('adb shell', shell=True, stdin=subprocess.PIPE)
communicate_nowait(proc_1, b'second adb command')
communicate_nowait(proc_2, b'second adb command')

Please note that you will have to take care of cleanup properly (refer to communicate documentation).

Warning: It seems that a too small timeout might cause problems (see reported bug).

Upvotes: 1

Related Questions