CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
mutantChickenHer0
mutantChickenHer0

Reputation: 223

BeautifulSoup - check for attribute / if no attribute

I have a function that scrapes information from a list of input URL's.

def scraper(inputlist):
    for url in inputlist:
        fullurl = baseurl + url
        hotelresponse = requests.get(fullurl)
        hotelsoup = BeautifulSoup(hotelresponse.text, "lxml")
        hoteltitle = hotelsoup.find('div', attrs={'class': 'vcard'})
        hotelhighprice = hotelsoup.find('div', attrs={'class': 'pricing'}).text
        for H1 in hoteltitle:
            hotelName = hoteltitle.find('h1').text
            time.sleep(2)
    return (hotelName, hotelhighprice, fullurl)

In this particular case, "hotelhighprice" may not always have a value.

I want to

A) If hotelhighprice is there / has a value, I want to return it. If not, then print a string "empty".

then, to iterate on that

B) If hotelhighprice is not there, look for a different value (that I will specify as a variable.

Current error message - 

  File "main.py", line 35, in scraper
    hotelhighprice = hotelsoup.find('div', attrs={'class': 'pricing'}).text
AttributeError: 'NoneType' object has no attribute 'text'

Upvotes: 1

Views: 4322

Answers (3)

Michał Zaborowski
Michał Zaborowski

Reputation: 4387

a = hotelsoup.find('div', attrs={'class': 'pricing'}) 
if a is None:
  # no pricing
else:
  price = a.text

Upvotes: 1

dnit13
dnit13

Reputation: 2496

You can use 

text_value = getattr(hotelsoup.find('div', attrs={'class': 'pricing'}), "text", my_default_value)

Upvotes: 4

alecxe
alecxe

Reputation: 473763

A common code pattern is to check if what find() returns is "truthy":

price_elm = hotelsoup.find('div', attrs={'class': 'pricing'})
hotelhighprice = price_elm.get_text() if price_elm else "Empty"

Or, in an expanded form:

price_elm = hotelsoup.find('div', attrs={'class': 'pricing'})
if price_elm:
    hotelhighprice = price_elm.get_text() 
else: 
   hotelhighprice = "Empty"
   # or you may find a different element here
   # hotelhighprice = hotelsoup.find('div', class_="someotherclass").get_text()

Upvotes: 1

Related Questions