CODEWITHSUNDEEP
pythonbeautifulsoup
Wayne Lin
Wayne Lin

Reputation: 13

Can't not get content from URL with python

I'm trying get content from an Chinese web page with python and BeautifulSoup.However when i print the result,i get nothing in the console.So,can anybody tell me why?(ps:i also change with some other web pages,sometimes the code works,but sometimes it doesn't) Here is my code:

# _*_ coding:utf-8 _*_
from bs4 import BeautifulSoup
import urllib2
import urllib
import urllib2

url='http://finance.sina.com.cn/chanjing/cyxw/2015-12-17/doc-ifxmttcn4893506.shtml'
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
headers = { 'User-Agent' : user_agent }
try:
  request=urllib2.Request(url)
  response=urllib2.urlopen(request)
  html=response.read()
  content=BeautifulSoup(response)
  print content
except  urllib2.URLError,e:
   if hasattr(e,"code"):
      print e.code
   if hasattr(e,"reason"):
      print e.reason

Here is my result: enter image description here`

Upvotes: 1

Views: 181

Answers (2)

Learner
Learner

Reputation: 357

Try this :

page = requests.get('http://finance.sina.com.cn/chanjing/cyxw/2015-12-17/doc-ifxmttcn4893506.shtml')
print page.text
soup = BeautifulSoup(page.text)
soup.prettify()
print soup

Upvotes: 0

Patrick Fournier
Patrick Fournier

Reputation: 688

Try 

content=BeautifulSoup(html)

Upvotes: 2

Related Questions