CODEWITHSUNDEEP
pythonmultithreadingpython-2.7
mehdi
mehdi

Reputation: 9502

python thread exception cause stop the process

in below code if i change one of the url to something invalid the whole process will stop and i couldn't exit form terminal using ctrl+c . so my question is how should i handle exception in my main thread run method and if an error happen trigger it and go to the next list element without fail the whole process: 

#!/usr/bin/env python
import Queue
import threading
import urllib2
import time

hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com","http://apple.com"]

queue = Queue.Queue()

class ThreadUrl(threading.Thread):
    """Threaded Url Grab"""
    def __init__(self, queue):
        threading.Thread.__init__(self)
        self.queue = queue

    def run(self):
        while True:
            #grabs host from queue
            host = self.queue.get()

            #grabs urls of hosts and prints first 1024 bytes of page
            url = urllib2.urlopen(host)
            print "connected"

            #signals to queue job is done
            self.queue.task_done()


start = time.time()
def main():

    #spawn a pool of threads, and pass them queue instance 
    for i in range(5):
        t = ThreadUrl(queue)
        t.setDaemon(True)
        t.start()


    #populate queue with data
    for host in hosts:
        queue.put(host)

    #wait on the queue until everything has been processed
    queue.join()
main()
print "Elapsed Time: %s" % (time.time() - start)

Upvotes: 1

Views: 1048

Answers (1)

vikki
vikki

Reputation: 2814

use a finally block to make sure the thread always signals even when there is an error.

def run(self):
    while True:
        #grabs host from queue
        host = self.queue.get()

        #grabs urls of hosts and prints first 1024 bytes of page
        try:
            url = urllib2.urlopen(host)
            print "connected"
        except urllib2.URLError:
            print "couldn't connect to %s" % host

        finally:
            #signals to queue job is done
            self.queue.task_done()

Upvotes: 3

Related Questions