CODEWITHSUNDEEP
pythonjsonsslhttpsurllib2
zch
zch

Reputation: 3060

How can I go about getting the JSON from a https link using Python?

I am learning Python, and I want to start using Twitter's streaming API. I know how to build a very basic application that reads the contents of a page using urllib2 and have done this successfully in the past. However, the secure connection https:// that this particular Twitter streaming API requires doesn't appear to work with urllib2.

I have tried to fetch the JSON from the following link: https://stream.twitter.com/1.1/statuses/filter.json?locations=-122.75,36.8,-121.75,37.8

By writing the following:

import urllib2
url = "https://stream.twitter.com/1.1/statuses/filter.json?locations=-122.75,36.8,-121.75,37.8"
print urllib2.urlopen(url).read()

The traceback, though, says I am not authorized; I presumed this is because of the HTTPS SSL connection, and research supported this. So I guess my question is what is the best way to go about getting the JSON from a link requiring such SSL connection? I also tried curl, but only get error 401 again.

Below is the traceback in its entirety.

Traceback (most recent call last): File "", line 1, in File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 126, in urlopen File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 400, in open File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 513, in http_response File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 438, in error File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 372, in _call_chain File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 521, in http_error_default urllib2.HTTPError: HTTP Error 401: Unauthorized

Many thanks in advance.

Upvotes: 0

Views: 1921

Answers (2)

Purrell
Purrell

Reputation: 12871

As others mention, you need to authorize. Here's how to do it:

See Twitter API with urllib2 in python

import urllib2
import base64

request = urllib2.Request( 'https://stream.twitter.com...' )
request.add_header( 'Authorization', 'Basic ' + base64.b64encode( username + ':' + password ) )
response = urllib2.urlopen( request )

Upvotes: 2

Xymostech
Xymostech

Reputation: 9850

It's not the SSL connection that's not authorizing, its that you haven't explicitly authorized with the system before making this call. You need to perform some sort of authorization with the server before making your request.

Upvotes: 1

Related Questions