CODEWITHSUNDEEP
pythonexceptionweb-scrapingattributeerror
Darkoder
Darkoder

Reputation: 53

How to make a long list of exceptions shorter in Python?

I am new to programming and I am trying to handle errors on my web scraping program. I iterate from a product list through 3 websites (A, B and C) looking for the product's name and product's price. For example i want this output:

print(productA, priceA, productB, priceB, productC, price C)

But sometimes some products do not store let say the price or the products name because it maybe be out of stock or it just can't find it and brings an AttributeError.

Because of this, I add a long list of exceptions on my program to print "not available" in each case it can't find the item's name or price it is looking for.

try:
    print(productA, priceA, productB, priceB, productC, price C)

except AttributeError:

   try:        
       print("not available", priceA, productB, priceB, productC, price C)

   except AttributeError:

       try:        
           print(productA, "not available", productB, priceB, productC, price C)

       except AttributeError:

           try:        
               print("not available", "not available", productB, priceB, productC, price C)
...

And so on for the three products, trying to see if one, two, or the three items' name or price may be missing and bringing up the error. My question is, is there a way to make this easier/faster or automate it so the code won't be so long? Thanks

Upvotes: 0

Views: 408

Answers (3)

Jeff Learman
Jeff Learman

Reputation: 3287

Consider factoring out these aspects:

  • name of item you're searching for
  • site you're searching (keep in mind you may add more sites later!)
  • name you find at a given site (if any)
  • price you find at a given site (if any)

Something like this:

products = [
    "LED flashlight",
    "AAA battery",
    "AA battery"
]

sites = {
    "Amazon" : "http://amazon.com",
    "Ebay" : "http://ebay.com",
    "Monoprice": "http:monoprice.com"
}

def my_search(url, prodname):
    # your site search code here
    # ...
    return (foundname, foundprice)

for product in products:
    for (site, url) in sites.items():
        (name, price) = my_search(url, product)
            try:
                print(name, end=' ')
            except Exception:
                print("not available", end=' ')
            try:
                print(price, end=' ')
            except Exception:
                print("not available", end=' ')
        print()

With minor changes you could easily produce an HTML or table or CSV file including column headers, that is easily modified to add sites or products. Note that my regex patterns above are NOT good regex examples!

Finally, it's a bad idea to catch Exception for logic purposes, because then you will mishandle other kinds of errors (like someone trying to abort the program if it's caught in an infinite loop.) Find out which exception you're getting and catch just that exception. Better yet, have your search method produce None or "not available" for product name or price, if it doesn't find them.

Upvotes: 0

vash_the_stampede
vash_the_stampede

Reputation: 4606

This is demonstration with pseudo values and using ValueError the values I assigned are irrelevant and only used to raise an Error to show how this could be done with a loop. In your code it would be AttributeError if you put all your items in a list then used a loop to print each item, and you could then just print an alternate message if the error caught. If you want everything on one line you could just add end = ' ' to your print statements

productA, priceA, priceB, productC, priceC = '1', '10', 'blah', '10', '100'

lista = [productA, priceA, priceB, productC, priceC]
for i in lista:
    try:
        print(int(i))
    except ValueError:
        print('{} is not available'.format(i))

1
10
blah is not available
10
100

Upvotes: 0

mad_
mad_

Reputation: 8273

Simple example of implementing dictionary and iterating it over

d={'product A':'', 'product B':22, 'product C':33}

for key,value in d.items():
    try:
        print('product:{}, price:{}'.format(key,int(value)))
    except Exception:
        print('price for {} is not available'.format(key))

Upvotes: 1

Related Questions