CODEWITHSUNDEEP
pythonmultithreadingwxpythonmultiprocessingnonblocking
RMWChaos
RMWChaos

Reputation: 885

wxPython non-blocking GUI threading AND multiprocessing?

Python 2.7.3 x64 wxPython 2.8 x64

Been reading quite a bit on python threading and multiprocessing, particularly some articles by Doug Hellmann, which have helped tremendously. However, I'm confused about one thing...

I thought the Python multiprocessing module was more-or-less a drop-in replacement for the threading module, excepting that args must be picklable, but I'm finding that in order not to block my GUI, I must first create a new thread with threading.Thread then multiprocess within that thread with multiprocessing.Process. This works, and works well, but it seems a bit kludgey to me.

If I try to directly multiprocess without first threading, then my GUI still blocks until the multiprocessing job is done. Is that working as designed, or am I missing something fundamental about the multiprocessing module?

If examples are needed, I can provide them.

Thanks,

-RMWChaos

An example was requested...

Assuming that onProcess() is triggered by a button in the GUI, this blocks the GUI...

import time
import multiprocessing as mp

def myWorker(a, b):
    time.sleep(0.1)
    print '{} * {} = {}'.format(a, b, a*b)

def onProcess(event):
    jobs = mp.cpu_count() * 2
    a = 5
    b = 10

    for job in range(jobs):
        mp.Process(target = myWorker, args = (a, b,)).start()

While this doesn't...

import time
import multiprocessing as mp
import threading as th

def myWorker(a, b):
    time.sleep(0.1)
    print '{} * {} = {}'.format(a, b, a*b)

def onProcess(event):
    a = 5
    b = 10
    th.Thread(target = myThread, args = [a, b,]).start()

def myThread(a, b):
    jobs = mp.cpu_count() * 2

    for job in range(jobs):
        mp.Process(target = myWorker, args = (a, b,)).start()

Upvotes: 1

Views: 5524

Answers (2)

Amr
Amr

Reputation: 735

I tried running your code in the following test program and multiprocessing works fine, nothing is blocked:

import time
import multiprocessing as mp
import wx


def myWorker(a, b):
    time.sleep(10)
    print '{} * {} = {}'.format(a, b, a*b)

def onProcess(event):
    jobs = mp.cpu_count() * 2
    a = 5
    b = 10

    for job in range(jobs):
        mp.Process(target = myWorker, args = (a, b,)).start()

def onGUI(event):
    print 'GUI is not blocked'

class MyFrame(wx.Frame):
    def __init__(self, parent, id, title):
       wx.Frame.__init__(self, parent, id, title)
       buttons = []
       panel = wx.Panel(self, wx.ID_ANY)
       sizer = wx.BoxSizer(wx.VERTICAL)
       gui_proc_btn = wx.Button(panel, wx.ID_ANY, 'GUI Process')
       other_proc_btn = wx.Button(panel, wx.ID_ANY, 'Other process')

       gui_proc_btn.Bind(wx.EVT_BUTTON, onGUI)
       sizer.Add(gui_proc_btn, 0, wx.ALL, 5)
       other_proc_btn.Bind(wx.EVT_BUTTON, onProcess)
       sizer.Add(other_proc_btn, 0, wx.ALL, 5)
       panel.SetSizer(sizer)

class MyApp(wx.App):
    def OnInit(self):
        frame = MyFrame(None, -1, 'test.py')
        frame.Show(True)
        self.SetTopWindow(frame)
        return True

if __name__ == '__main__':

    app = MyApp(0)
    app.MainLoop()

Run this from your command-line, press the second button(this uses multiprocessing on your function, and I increased the sleep time), and then press the first button. you should notice that there's output generated when pressing the first button so the program is not blocked.

Upvotes: 1

forTruce
forTruce

Reputation: 849

Multi-threading and multi-processing are fundamentally different.

Threads are used for i/o for the most part. When you make a new thread it is contained within the same process as the program you spawned the thread from. What that means is that it shares memory space with the program, but they (the program and the thread) cannot run in parallel (also look up GIL).

Multi-processing on the other hand spawns a new process on the OS level. This new process can run in parallel with the pre-existing process, but it does not share memory space with the program that spawned it. This is more useful when the code you want to speed up is not i/o related but actual processor intensive computation.

For a gui, you mostly want to use threading for different parts of the gui so that running something in one part of the gui does not lock up your entire gui until that handling ends.

Without code it is hard to tell, but I believe you should not particularly need to spawn a process within the new thread, but instead just have that thread handle the processing. However, if that thread needs to handle intensive computational type of processing as opposed to lots of i/o, then you would want to start a process to handle that.

I believe that most of your problem lies in misunderstanding multi-processing vs multithreading.

Upvotes: 5

Related Questions