How to handle Python's threading.Thread exceptions?

Question

I'm using Selenium to run two Firefox instances and get specifics pages. I'm loading the pages parallel using threading.Thread for each of them. I also want to set timeout for maximum page loading time, with browser.set_page_load_timeout() in my code.

My whole code looks like this:

from selenium import webdriver
from threading import Thread
from selenium.common.exceptions import TimeoutException


class Test():
        def __init__(self):
            browser = webdriver.Firefox()

            def load_page(browser, url):
                browser.set_page_load_timeout(20)
                browser.get(url)

            t = Thread(target=load_page, args=(browser, 'http://www.stackoverflow.com', ))
            t.start()
            t.join()

if __name__ == '__main__':
    try:
        Test()
    except TimeoutException:
        print "timeout reached"

In spite of my try except declaration, i still got this error:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Python27\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Python27\lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "C:\Temp\test_b.py", line 13, in load_page
    browser.get(url)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 213, in get
    self.execute(Command.GET, {'url': url})
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 201, in execute
    self.error_handler.check_response(response)
  File "C:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 181, in check_response
    raise exception_class(message, screen, stacktrace)
TimeoutException: Message: Timed out waiting for page load.
Stacktrace:
    at Utils.initWebLoadingListener/< (file:///c:/users/mgal/appdata/local/temp/tmpsasxck/extensions/fxdriver@googlecode.com/components/driver-component.js:9010)
    at WebLoadingListener/e (file:///c:/users/mgal/appdata/local/temp/tmpsasxck/extensions/fxdriver@googlecode.com/components/driver-component.js:5114)
    at WebLoadingListener/< (file:///c:/users/mgal/appdata/local/temp/tmpsasxck/extensions/fxdriver@googlecode.com/components/driver-component.js:5122)
    at fxdriver.Timer.prototype.setTimeout/<.notify (file:///c:/users/mgal/appdata/local/temp/tmpsasxck/extensions/fxdriver@googlecode.com/components/driver-component.js:621)

In conclusion, how can I catch the Timeout Exception out of the thread scope? Thanks ahead!

Answer 1

You can't. You should handle exception in the thread, so your function should look more or less like:

def load_page(browser, url):
    try:
        browser.set_page_load_timeout(20)
        browser.get(url)
    except TimeoutException:
        '''Handle me here'''

EDIT: What you actually request for is:

from selenium import webdriver
from threading import Thread
from Queue import Queue
from selenium.common.exceptions import TimeoutException


class Test():
        def __init__(self, queue, url):
            browser = webdriver.Firefox()

            def load_page(browser, url):
                try:
                    browser.set_page_load_timeout(20)
                    browser.get(url)
                except Exception as e:
                    queue.put(e)
                else:
                    queue.put('OK or whatever the result you want')

            t = Thread(target=load_page, args=(browser, url, ))
            t.start()

if __name__ == '__main__':
    urls = ('http://www.stackoverflow.com', 'http://http://meta.stackoverflow.com/')
    queue = Queue()
    for url in urls:
        Test(queue, url)
    for i in range(len(urls)):
        result = queue.get()
        if isinstance(result, Exception):
            '''Handle exception preferably trying to determine the actual exception type'''
        else:
            '''Say cool cause everything is fine'''

Answer 2

Threading children run in their own stack, so this is impossible without message/event passing. You can use python's Queue library (which is thread-safe) and pass a queue object into your child function, using that as an event pool that the parent can handle.