CODEWITHSUNDEEP
pythonpython-3.xmultiprocessingpython-multiprocessingpython-multithreading
randy
randy

Reputation: 166

Nothing executes in the main process after the last child process is started, why?

I have this code here:

if __name__ == '__main__':

    #iterate over markets_to_track
    for index, market in enumerate(markets_to_track):

        print('connecting to ', market['exmp'])

        #launch a child process for each market
        markets_to_track[index]['multiprocessing_pid'] = Process(target=collect_data, kwargs=market)
        markets_to_track[index]['multiprocessing_pid'].start()

        print('just connected to ', market['exmp'])

     print('main process now does it\'s thing!')

The child processes execute fine... There are two markets_to_track and they both execute as expected.

The first print that says "connecting to market['exmp']" is printed twice.

The second print that says "just connected to market['exmp']" is only printed once (after the first iteration of the for loop) and nothing in the main process is executed thereafter.

print('main process now does it\'s thing!') never sees the light of day.

What am I missing?

Thanks for reading.

EDIT: Adding SyntaxVoid's suggestion of flush=True to the print statements has them printing now, but why?!

Upvotes: 2

Views: 72

Answers (1)

RaJa
RaJa

Reputation: 1567

Adding flush to the print-statement tells the interpreter to perform the output right now, acquiring the required CPU time ASAP. Otherwise the processing of the stream is scheduled for the moment, when ressources become available. As your sub-processes however constantly allocate CPU-time, the print-statement will not be executed until the sub-processes have finished (joined). So the output of the print appears at the very end of your program.

The same problem sometimes occur when doing lengthy calculations with status updates, like a progress bar during copying files.

Upvotes: 1

Related Questions