CODEWITHSUNDEEP
pythonjsonurllib2
Deepak B
Deepak B

Reputation: 2335

Python urllib2: Receive JSON response from url

I am trying to GET a URL using Python and the response is JSON. However, when I run

import urllib2
response = urllib2.urlopen('https://api.instagram.com/v1/tags/pizza/media/XXXXXX')
html=response.read()
print html

The html is of type str and I am expecting a JSON. Is there any way I can capture the response as JSON or a python dictionary instead of a str.

Upvotes: 101

Views: 184800

Answers (10)

Himanshu Aggarwal
Himanshu Aggarwal

Reputation: 179

This is another simpler solution to your question

pd.read_json(data)

where data is the str output from the following code

response = urlopen("https://data.nasa.gov/resource/y77d-th95.json")
json_data = response.read().decode('utf-8', 'replace')

Upvotes: 0

Haritsinh Gohil
Haritsinh Gohil

Reputation: 6272

you can also get json by using requests as below:

import requests

r = requests.get('http://yoursite.com/your-json-pfile.json')
json_response = r.json()

Upvotes: 1

Adam
Adam

Reputation: 2897

Python 3 standard library one-liner:

load(urlopen(url))

# imports (place these above the code before running it)
from json import load
from urllib.request import urlopen
url = 'https://jsonplaceholder.typicode.com/todos/1'

Upvotes: 1

SanalBathery
SanalBathery

Reputation: 638

import json
import urllib

url = 'http://example.com/file.json'
r = urllib.request.urlopen(url)
data = json.loads(r.read().decode(r.info().get_param('charset') or 'utf-8'))
print(data)

urllib, for Python 3.4
HTTPMessage, returned by r.info()

Upvotes: 44

alphojo
alphojo

Reputation: 630

"""
Return JSON to webpage
Adding to wonderful answer by @Sanal
For Django 3.4
Adding a working url that returns a json (Source: http://www.jsontest.com/#echo)
"""

import json
import urllib

url = 'http://echo.jsontest.com/insert-key-here/insert-value-here/key/value'
respons = urllib.request.urlopen(url)
data = json.loads(respons.read().decode(respons.info().get_param('charset') or 'utf-8'))
return HttpResponse(json.dumps(data), content_type="application/json")

Upvotes: 5

Nitigya Sharma
Nitigya Sharma

Reputation: 63

Though I guess it has already answered I would like to add my little bit in this

import json
import urllib2
class Website(object):
    def __init__(self,name):
        self.name = name 
    def dump(self):
     self.data= urllib2.urlopen(self.name)
     return self.data

    def convJSON(self):
         data=  json.load(self.dump())
     print data

domain = Website("https://example.com")
domain.convJSON()

Note : object passed to json.load() should support .read() , therefore urllib2.urlopen(self.name).read() would not work . Doamin passed should be provided with protocol in this case http

Upvotes: 0

Uxbridge
Uxbridge

Reputation: 429

None of the provided examples on here worked for me. They were either for Python 2 (uurllib2) or those for Python 3 return the error "ImportError: No module named request". I google the error message and it apparently requires me to install a the module - which is obviously unacceptable for such a simple task.

This code worked for me: 

import json,urllib
data = urllib.urlopen("https://api.github.com/users?since=0").read()
d = json.loads(data)
print (d)

Upvotes: -1

Jossef Harush Kadouri
Jossef Harush Kadouri

Reputation: 34257

resource_url = 'http://localhost:8080/service/'
response = json.loads(urllib2.urlopen(resource_url).read())

Upvotes: 2

Martijn Pieters
Martijn Pieters

Reputation: 1124858

If the URL is returning valid JSON-encoded data, use the json library to decode that:

import urllib2
import json

response = urllib2.urlopen('https://api.instagram.com/v1/tags/pizza/media/XXXXXX')
data = json.load(response)   
print data

Upvotes: 194

MostafaR
MostafaR

Reputation: 3695

Be careful about the validation and etc, but the straight solution is this:

import json
the_dict = json.load(response)

Upvotes: 4

Related Questions