CODEWITHSUNDEEP
pythonurllib2
zachary
zachary

Reputation: 3

urllib2 Error 403: Forbidden

I have posted to this site and received really helpful guidance, i return with another question.

Where have i gone wrong here, I was prettty sure this is what is required to access information from various sites. In this case, the CME Group.

   import urllib2

url = "http://www.cmegroup.com/trading/energy/natural-gas/natural-gas.html"
request= urllib2.Request(url)
handle = urllib2.urlopen(request)
content = handle.read()
splitted_page = content.split("<span class=\"cmeSubHeading\">", 1);
splitted_page = splitted_page[1].split("</span>", 1)
print splitted_page[0]

Error reads, 

HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

Thank you greatly in advance.

Upvotes: 0

Views: 1475

Answers (2)

pypy
pypy

Reputation: 463

If you have to do GET requests I reccomend you the Requests python package. You can read its advantages in this post.

However, if you're getting a 403 message maybe you're trying to access some restricted data (Wikipedia link).

Upvotes: 1

Cake
Cake

Reputation: 81

Actually the problem is that they block everyone who doesn't have a user-agent

import urllib2

request = urllib2.Request("http://www.cmegroup.com/trading/energy/natural-gas/natural-gas.html", None, {'User-Agent': 'Mozilla/5.0'})
content = urllib2.urlopen(request).read()
splitted_page = content.split("<span class=\"cmeSubHeading\">", 1);
splitted_page = splitted_page[1].split("</span>", 1)
print splitted_page[0]

Upvotes: 1

Related Questions