CODEWITHSUNDEEP
pythonpython-3.xmultiprocessingthreadpool
Ijaz Ur Rahim
Ijaz Ur Rahim

Reputation: 410

Python - Handle Exception inside multiprocessing Pool.map

I have the following Code, But when i try it raise those errors that i even handled with try except 

from multiprocessing.dummy import Pool as ThreadPool 
def getPrice(product='',listing=False):
    try:
        avail = soup.find('div',id='availability').get_text().strip()
    except:
        avail = soup.find('span',id='availability').get_text().strip()

pool.map(getPrice, list_of_hashes)

It gives me the following error

Traceback (most recent call last):
  File "C:\Users\Anonymous\Desktop\Project\google spreadsheet\project.py", line 4, in getPrice
    avail = soup.find('div',id='availability').get_text().strip()
AttributeError: 'NoneType' object has no attribute 'get_text'

Upvotes: 0

Views: 393

Answers (1)

Jean-François Fabre
Jean-François Fabre

Reputation: 140168

avail = soup.find('span',id='availability').get_text().strip() is inside the except statement, so it's not handled inside your function

Better loop on the properties and return a default value if not found:

def getPrice(product='',listing=False):
    for p in ['div','span']:
      try:
         # maybe just checking for not None would be enough
         avail = soup.find(p,id='availability').get_text().strip()
         # if no exception, break
         break
      except Exception:
        pass
    else:
        # for loop ended without break: no value worked
        avail = ""
    # don't forget to return your value...
    return avail

Upvotes: 1

Related Questions