CODEWITHSUNDEEP
pythonhtmlbeautifulsoupmeta
stackoverflow1234
stackoverflow1234

Reputation: 273

Extracting data from a meta tag beautifulsoup returning null

I trying to scrape the mileage of a car from this website https://cazana.com/uk/car/RA51GZJ

The data I want is the mileage (128,375 miles) when i try to scrape this page i get nothing returned I orignally tried to scape the body of the page with no luck

url = "https://cazana.com/uk/car/RA51GZJ"
page2 = requests.get(url)
soup2 = BeautifulSoup(page2.content, 'html.parser')
result = soup2.findAll('meta', attrs={'name': 'description'})

print (result)

Returns []

This is the html file

 <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="description" content="RA51GZJ - 2001 NISSAN ALMERA. Colour silver, 128,375 miles, 3 previous owners. Registered in Reading. Tax, MOT &amp; Vehicle history check available.">

Thanks

Upvotes: 1

Views: 1521

Answers (1)

Jeremiah
Jeremiah

Reputation: 633

Your request is unsuccessful, which is why you're not finding the right tag.
The returned content is an error page.
You can bypass this error by changing your User-Agent header to that of a browser:

import requests
from bs4 import BeautifulSoup

url = 'https://cazana.com/uk/car/RA51GZJ'

headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64)'
    'AppleWebKit/537.36 (KHTML, like Gecko)'
    'Chrome/64.0.3282.167 Safari/537.36'
}

result = requests.get(url, headers=headers)
soup = BeautifulSoup(result.content, 'html.parser')
match = soup.find('meta', name='description')

if match:
    print(match.attrs['content'])
else:
    print('Request unsuccessful')

Note that too many requests at once can also trigger an unsuccessful request.

Upvotes: 3

Related Questions