CODEWITHSUNDEEP
pythonbeautifulsouppython-requests
yogkm
yogkm

Reputation: 649

cannot run BeautifulSoup using requests.get(url)

start_url=requests.get('http://www.delicious.com/golisoda')
soup=BeautifulSoup(start_url)

this code is displaying the following error:

Traceback (most recent call last):
  File "test2_requests.py", line 10, in <module>
    soup=BeautifulSoup(start_url)
  File "/usr/local/lib/python2.7/dist-packages/bs4/__init__.py", line 169, in __init__
    self.builder.prepare_markup(markup, from_encoding))
  File "/usr/local/lib/python2.7/dist-packages/bs4/builder/_lxml.py", line 68, in prepare_markup
    dammit = UnicodeDammit(markup, try_encodings, is_html=True)
  File "/usr/local/lib/python2.7/dist-packages/bs4/dammit.py", line 203, in __init__
    self._detectEncoding(markup, is_html)
  File "/usr/local/lib/python2.7/dist-packages/bs4/dammit.py", line 373, in _detectEncoding
    xml_encoding_match = xml_encoding_re.match(xml_data)
TypeError: expected string or buffer

Upvotes: 1

Views: 2292

Answers (3)

Martijn Pieters
Martijn Pieters

Reputation: 1121524

Use the .content of the response:

start_url = requests.get('http://www.delicious.com/golisoda')
soup = BeautifulSoup(start_url.content)

Alternatively, you can use the decoded unicode text:

start_url = requests.get('http://www.delicious.com/golisoda')
soup = BeautifulSoup(start_url.text)

See the Response content section of the documentation.

Upvotes: 6

Fredrik Pihl
Fredrik Pihl

Reputation: 45644

from BeautifulSoup import BeautifulSoup
import urllib2
data=urllib2.urlopen('http://www.delicious.com/golisoda').read()
soup=BeautifulSoup(data)

Upvotes: 0

akhter wahab
akhter wahab

Reputation: 4085

you probebly need to Use

using 

soup=BeautifulSoup(start_url.read())

or 

soup=BeautifulSoup(start_url.text)

Upvotes: 0

Related Questions