CODEWITHSUNDEEP
pythonmultithreading
Dragon.M
Dragon.M

Reputation: 277

Python: How to add new thread after one of threads breaks on error

I'm trying to create loop of threads and so far code is good. But I have problem when thread exits because of some exception.

Now I'm trying to figure out how to start additional thread after one thread exits because of exception. I did browse around but I didn't find any example that would work for this complex code. Any help would be great!

If thread stopped and queue is not empty restart stopped thread and continue with rest of the list.

This is my code:

some_list = [1,2,3,4,5,6,7,8]
exitFlag = 0
class threads():
    @staticmethod
    def process_data(threadName, q,queueLock):
        workQueue = q
        while not exitFlag:
            queueLock.acquire()
            if not workQueue.empty():
                data = q.get()
                queueLock.release()
                print "%s processing %s" % (threadName, data)
            else:
                queueLock.release()
            sleep(1)

    def run_threads(self):
        threadList = ["Thread-1", "Thread-2", "Thread-3"]
        nameList = some_list
        queueLock = threading.Lock()
        workQueue = Queue.Queue(1000000)
        threads = []
        threadID = 1

        # Create new threads
        for tName in threadList:
            thread = myThread(threadID, tName, workQueue,queueLock)
            thread.start()
            threads.append(thread)
            threadID += 1

        # Fill the queue
        queueLock.acquire()
        for word in nameList:
            workQueue.put(word)
        queueLock.release()

        # Wait for queue to empty
        while not workQueue.empty():
            pass

        # Notify threads it's time to exit
        global exitFlag
        exitFlag = 1

        # Wait for all threads to complete
        for t in threads:
            t.join()
        print "Exiting Main Thread"


class myThread (threading.Thread,threads):
    def __init__(self, threadID, name, q,queueLock):
        self.thread = threading.Thread(target=self.run)
        threading.Thread.__init__(self,target=self.run)
        self.threadID = threadID
        self.queueLock = queueLock
        self.name = name
        self.q = q

    def run(self):
       print "Starting " + self.name
       threads.process_data(self.name, self.q,self.queueLock)
       print "Exiting " + self.name

threads().run_threads()

Upvotes: 2

Views: 224

Answers (1)

K. Kirsz
K. Kirsz

Reputation: 1420

Something like this should work:

...
    # Wait for queue to empty
    while not workQueue.empty():
        for (i, t) in enumerate(threads):
            if not t.is_alive():
                print("Recreating thread " + t.name)
                thread = myThread(threadID, threadList[i], workQueue,queueLock)
                thread.start()
                threads[i] = thread
                threadID += 1
...

I would advice putting the thread-starting code into some method, as it will now be duplicated and hard to maintain.

The problem here is that you might "loose" the data that was popped from queue by the fatal thread.

Upvotes: 1

Related Questions