CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
user1922698
user1922698

Reputation: 41

Finding specific text using BeautifulSoup

I'm trying to grab all the winner categories from this page: http://www.chicagoreader.com/chicago/BestOf?category=4053660&year=2013

I've written this in sublime:

import urllib2
from bs4 import BeautifulSoup
url = "http://www.chicagoreader.com/chicago/BestOf?category=4053660&year=2013"
page = urllib2.urlopen(url)
soup_package = BeautifulSoup(page)
page.close()

#find everything in the div class="bestOfItem). This works.
all_categories = soup_package.findAll("div",class_="bestOfItem")
# print(all_categories)

#this part breaks it:
soup = BeautifulSoup(all_categories)
winner = soup.a.string
print(winner)

When I run this in terminal, I get the following error:

Traceback (most recent call last):
  File "winners.py", line 12, in <module>
    soup = BeautifulSoup(all_categories)
  File "build/bdist.macosx-10.9-intel/egg/bs4/__init__.py", line 193, in __init__
  File "build/bdist.macosx-10.9-intel/egg/bs4/builder/_lxml.py", line 99, in prepare_markup
  File "build/bdist.macosx-10.9-intel/egg/bs4/dammit.py", line 249, in encodings
  File "build/bdist.macosx-10.9-intel/egg/bs4/dammit.py", line 304, in find_declared_encoding
TypeError: expected string or buffer

Any one know what's happening there?

Upvotes: 0

Views: 85

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1125058

You are trying to create a new BeautifulSoup object from a list of elements.

soup = BeautifulSoup(all_categories)

There is absolutely no need to do this here; just loop over each match instead:

for match in all_categories:
    winner = match.a.string
    print(winner)

Upvotes: 2

Related Questions