CODEWITHSUNDEEP
pythonbeautifulsoupcss-selectors
Moondra
Moondra

Reputation: 4511

soup.find_all works but soup.select doesn't work

I'm playing around with parsing an html page using css selectors 

import requests
import webbrowser
from bs4 import BeautifulSoup

page = requests.get('http://www.marketwatch.com',  headers={'User-Agent': 'Mozilla/5.0'})
soup = BeautifulSoup(page.content, 'html.parser')

I'm having trouble selecting a list tag with a class when using the select method. However, I have no trouble when using the find_all method

soup.find_all('ul', class_= "latestNews j-scrollElement")

This returns the output I desire, but for some reason I can't do the same using css selectors. I want to know what I'm doing wrong.

Here is my attempt:

soup.select("ul .latestNews j-scrollElement")

which is returning an empty list.

I can't figure out what I'm doing wrong with the select method.

Thank you.

Upvotes: 4

Views: 1863

Answers (1)

cs95
cs95

Reputation: 402523

From the documentation:

If you want to search for tags that match two or more CSS classes, you should use a CSS selector:

css_soup.select("p.strikeout.body")

In your case, you'd call it like this:

In [1588]: soup.select("ul.latestNews.j-scrollElement")
Out[1588]: 
[<ul class="latestNews j-scrollElement" data-track-code="MW_Header_Latest News|MW_Header_Latest News_Facebook|MW_Header_Latest News_Twitter" data-track-query=".latestNews__headline a|a.icon--facebook|a.icon--twitter">
 .
 .
 .

Upvotes: 4

Related Questions