CODEWITHSUNDEEP
pythonpython-3.xwebweb-scrapingbeautifulsoup
arthem
arthem

Reputation: 151

Best way to scrape data in Python from this website?

I am trying to scrape the price of the coin from the link below using the following Python Script, however I am running into some issues. If you could identify where I am going wrong then that would be great!

URL's for various coins from various websites are stored in a Python Dictionary called coins. A try, except, if stack is used to find the appropiate method to get the price. For some reason I am unable to exract the price from the following URL:

https://www.hattongardenmetals.com/buy/2020-gold-britannia-2

The script being used is:

    for coin in prices:
    response = requests.get(prices[coin]["url"])
    soup = BeautifulSoup(response.text, 'html.parser')

    try:
        text_price = soup.find(
            'td', {'id': 'price-inc-vat-per-unit-1'}).get_text()        
    except:
        text_price = soup.find(
            'td', {'id': 'total-price-inc-vat-1'}).get_text()            
    else:
        text_price = soup.find(
            'span', {'class': 'woocommerce-Price-amount amount'}).get_text       #<------

For some reason the line highlighted by the arrow keeps getting this error:

AttributeError: 'NoneType' object has no attribute 'get_text'

How can I fix this and how does you fix integrate with the code provided?

Upvotes: 0

Views: 373

Answers (2)

fardV
fardV

Reputation: 308

try putting a parenthesis .. after get_text text_price = soup.find('span', {'class': 'woocommerce-Price-amount amount'}).get_text()

Upvotes: 1

Andrej Kesely
Andrej Kesely

Reputation: 195428

The price is inside <span> tag with class="h2 d-block":

import requests
from bs4 import BeautifulSoup


url = 'https://www.hattongardenmetals.com/buy/2020-gold-britannia-2'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

print( soup.select_one('span.h2').text )

Prints:

£1584.31

Upvotes: 2

Related Questions