CODEWITHSUNDEEP
pythonbeautifulsoup
Kay
Kay

Reputation: 345

BeautifulSoup AttributeError: 'NoneType' object has no attribute 'text'

I am trying to learn how to scrape websites and therefore not using an API. I am trying to scrape eBay's websites and my script runs for a few minutes but stops because of an error stating "AttributeError: 'NoneType' object has no attribute 'text'". I did my due diligence and search on Google/StackOverflow help but was unable to find any. The error points to the code below but I cannot figure out how to fix it. 

total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text

if total_sold_price:
    total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text

else:
    try:
        total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
    except Exception as e:
        print(e)
        total_sold_price = ""

Entire code: https://pastebin.com/xS4bAwZK

Thanks in advance, I greatly appreciate it.

Upvotes: 1

Views: 1103

Answers (2)

Duda
Duda

Reputation: 1

Is it solved? Instead of {'class': 'vi-qtyS-hot-red'} You can write = 'vi-qtyS-hot-red'

No need write 'class' and {}

Upvotes: 0

chitown88
chitown88

Reputation: 28565

The issue is soup.find('span', {'class': 'vi-qtyS-hot-red'}) returns None, and then you are trying to extract the text .text of None (hence the error message). There are a few ways you can handle it. All I did was move around some logic:

Also, you have it store the text total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text, and if it stores it, you have it do the exact something which is redundant.

Lastly, the page is dynamic, so you might want to look into API or other ways to access the data.

But for your code provided, maybe something like this instead:

try:
    total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text
except Exception as e:
    try:
        total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
    except Exception as e:
                print(e)
                total_sold_price = ""

Upvotes: 2

Related Questions