CODEWITHSUNDEEP
pythonhtmlweb-crawler
Bluelily
Bluelily

Reputation: 63

My python code output wrong html data

I have this code snippet as part of python code to crawl a particular website (see the code below). But to my surprise the output code is not an html. I am using python 3.4

   import urllib.request as ur
   user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
   headers = { 'User-Agent' : user_agent }

   s = ur.urlopen('http://www.nairaland.com')
   pl = s.read()
   print(pl)

My output from this code is:

b''

rather than the expected html code. Please guide me towards getting this code work. I need the html code in another part of the code. Thanks in advance.

Upvotes: 1

Views: 80

Answers (1)

Robᵩ
Robᵩ

Reputation: 168626

The excellent requests library returns the correct HTML:

import requests
s = requests.get('http://www.nairaland.com')
pl = s.text
print(pl)

Upvotes: 1

Related Questions