CODEWITHSUNDEEP
pythonseleniumweb-scrapingbeautifulsoupscreen-scraping
Ryan
Ryan

Reputation: 33

Got empty list with Beautiful Soup and Selenium

https://www.rottentomatoes.com/m/the_lord_of_the_rings_the_return_of_the_king

I want to get TOMATOMETER and AUDIENCE SCORE from that website, but got an empty list.

soup = BeautifulSoup(html, 'html.parser')
notices = soup.select('#tomato_meter_link > span.mop-ratings-wrap__percentage')

Upvotes: 0

Views: 64

Answers (2)

QHarr
QHarr

Reputation: 84465

You can use last-child selector for span type with the parent class. This is using BeautifulSoup 4.7.1

import requests
from bs4 import BeautifulSoup

res = requests.get('https://www.rottentomatoes.com/m/the_lord_of_the_rings_the_return_of_the_king')
soup = bs(res.content, 'lxml')
ratings = [item.text.strip() for item in soup.select('h1.mop-ratings-wrap__score span:last-child')]
print(ratings)

Upvotes: 1

nmb.ten
nmb.ten

Reputation: 2228

Your code works well

>>> from bs4 import BeautifulSoup
>>> html = requests.get('https://www.rottentomatoes.com/m/the_lord_of_the_rings_the_return_of_the_king').text
>>> soup = BeautifulSoup(html, 'html.parser')
>>> notices = soup.select('#tomato_meter_link > span.mop-ratings-wrap__percentage')
>>> notices
[<span class="mop-ratings-wrap__percentage">93%</span>]

How did you get html variable?

Upvotes: 0

Related Questions