Why process.communicate() return tuple of NoneType?

Question

Below program, launches subprocess and communicate via pipes,

import json
import math
import subprocess
import sys
import time

if __name__ == '__main__':
    with open(sys.argv[1], 'r') as f:
        websites = f.read().splitlines()
    numberOfProcesses = int(sys.argv[2])
    perProcess = math.ceil(len(websites) / numberOfProcesses)
    for i in range(numberOfProcesses):
        sites = websites[i*perProcess:(i+1)*perProcess]
        with open("/tmp/list-{}.txt".format(i), "w") as f:
            f.write("\n".join(sites))
    t0 = time.time()
    processes = []
    for i in range(numberOfProcesses):
        p = subprocess.Popen(
            ["python3.6", "naive-checker.py", "/tmp/list-{}.txt".format(i)]
        )
        processes.append(p)
    # Gather the results
    for process in processes:
        result = process.communicate()
        print('result:',result)
    t1=time.time()
    print("Total time getting website status took {0:1f} seconds", format(t1-t0))

---

result shows (None, None) instead of (stdout, stderr), where each subprocess is,

import json
import requests
import sys
import time


def website_statuses(websites):
    statuses = {}
    for website in websites:
        response = requests.get(website)
        status = response.status_code
        if not statuses.get(status):
            statuses[status] = 0
        statuses[status] += 1
    return statuses


if __name__ == '__main__':
    with open(sys.argv[1], 'r') as f:
        websites = f.read().splitlines()
    t0 = time.time()
    print('From this subprocess, status code: ', json.dumps(website_statuses(websites)))
    t1 = time.time()
    print("For this subprocess, getting website statuses took {0:.1f} seconds".format(t1-t0))

---

list.txt is file of websites.

Output:

$ python3.6 dummy.py list.txt 3
 From this subprocess, status code:  {"200": 11}
 For this subprocess, getting website statuses took 5.3 seconds
 From this subprocess, status code:  {"200": 9}
 For this subprocess, getting website statuses took 6.8 seconds
 From this subprocess, status code:  {"200": 11}
 For this subprocess, getting website statuses took 9.8 seconds
 result: (None, None)
 result: (None, None)
 result: (None, None)
 Total time getting website status took {0:1f} seconds 10.265882730484009
$

---

Question:

Why result shows tuple of NoneType?

Answer 1

You should look at the documentation:

communicate() returns a tuple (stdout_data, stderr_data). The data will be strings if streams were opened in text mode; otherwise, bytes.

Note that if you want to send data to the process’s stdin, you need to create the Popen object with stdin=PIPE. Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.

As you did not specify any output pipe while creating your Popen object, the returned values are two None while calling communicate().

You may try to fix it with:

p = subprocess.Popen(
    ["python3.6", "naive-checker.py", "/tmp/list-{}.txt".format(i)],
     stdout=subprocess.PIPE, stderr=subprocess.PIPE
)

Answer 2

From Popen.communicate documentation:

Similarly, to get anything other than None in the result tuple, you need to give stdout=PIPE and/or stderr=PIPE too.