CODEWITHSUNDEEP
pythonipythonurllib
Collective Action
Collective Action

Reputation: 8009

TypeError parsing JSON obj from urllib ipython

I am using an API to request data from a website. The data can be found here and pasted into a JSON Viewer. My code along with the error that it is returning are below. I am guessing that this is a quick fix, partially reflecting the fact that this is my first time using urllib.

import pandas as pd
import urllib 
import json

api_key = '79bf8eb2ded72751cc7cda5fc625a7a7'
url = 'http://maplight.org/services_open_api/map.bill_list_v1.json?apikey=79bf8eb2ded72751cc7cda5fc625a7a7&jurisdiction=us&session=110&include_organizations=1&has_organizations=1'

json_obj = urllib.request.urlopen(url)

data = json.load(json_obj)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-21-85ab9af07320> in <module>()
      8 json_obj = urllib.request.urlopen(url)
      9 
---> 10 data = json.load(json_obj)

/home/jayaramdas/anaconda3/lib/python3.5/json/__init__.py in load(fp, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    266         cls=cls, object_hook=object_hook,
    267         parse_float=parse_float, parse_int=parse_int,
--> 268         parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
    269 
    270 

/home/jayaramdas/anaconda3/lib/python3.5/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    310     if not isinstance(s, str):
    311         raise TypeError('the JSON object must be str, not {!r}'.format(
--> 312                             s.__class__.__name__))
    313     if s.startswith(u'\ufeff'):
    314         raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",

TypeError: the JSON object must be str, not 'bytes'

Any suggestions, comments, or further questions are appreciated.

Upvotes: 0

Views: 224

Answers (1)

mgilson
mgilson

Reputation: 310167

json.load won't guess the encoding, so you typically need to .read the bytes from the object returned and then convert those bytes into a string by using .decode and the appropriate codec. e.g.:

data = json.loads(json_obj.read().decode('utf-8'))

There is an example of this in the official documentation.

Specifically, it says:

Note that urlopen returns a bytes object. This is because there is no way for urlopen to automatically determine the encoding of the byte stream it receives from the http server. In general, a program will decode the returned bytes object to string once it determines or guesses the appropriate encoding.

Upvotes: 3

Related Questions