user3675188
user3675188

Reputation: 7409

How to convert json response into Python list

I get the JSON response by requests.get

req = requests.get(SAMPLE_SCHEDULE_API)

and convert it into dictionary

data = json.loads(req.text)["data"]

When I tried to convert the string into Python dict,

I got ValueError: malformed node or string:

ast.literal_eval(data)

I have no idea how to do this task.

code snippets

    def schedules(cls, start_date=None, end_date=None):
        import ast
        req = requests.get(SAMPLE_SCHEDULE_API)
        data = json.loads(req.text)["data"]
        ast.literal_eval(data)
        return pd.DataFrame(json.loads(req.text)["data"])

JSON response

{
status: "ok",
version: "v1",
data: "[
{"_id":"2015-01-28","end_date":"2015-01-28","estimated_release":1422453600000,"is_projection":false,"is_statement":true,"material_link":null,"start_date":"2015-01-27"},
{"_id":"2015-03-18","end_date":"2015-03-18","estimated_release":1426687200000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-03-17"},
{"_id":"2015-04-29","end_date":"2015-04-29","estimated_release":1430316000000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-04-28"},
{"_id":"2015-06-17","end_date":"2015-06-17","estimated_release":1434549600000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-06-16"},
{"_id":"2015-07-29","end_date":"2015-07-29","estimated_release":1438178400000,"is_projection":false,"is_statement":false,"material_link":null,"start_date":"2015-07-28"}]"
}

Detail error message

Traceback (most recent call last):
  File "fomc.py", line 25, in <module>
    schedules = FOMC.schedules()
  File "fomc.py", line 21, in schedules
    ast.literal_eval(data)
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 86, in literal_eval
    return _convert(node_or_string)
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 58, in _convert
    return list(map(_convert, node.elts))
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/usr/local/Cellar/python3/3.3.2/Frameworks/Python.framework/Versions/3.3/lib/python3.3/ast.py", line 85, in _convert
    raise ValueError('malformed node or string: ' + repr(node))
ValueError: malformed node or string: <_ast.Name object at 0x10a19c990> 

Upvotes: 0

Views: 11549

Answers (3)

salmanwahed
salmanwahed

Reputation: 9647

As the response already is json format, you do not need to encode it. Approach like this,

req = requests.get(SAMPLE_SCHEDULE_API)
data_str = req.json().get('data')
json_data = json.loads(data_str)

json() method will return the json-encoded content of a response.

Upvotes: 1

You have encoded the data twice (which would strictly not be necessary). You just need to decode the data again with json.loads:

def schedules(cls, start_date=None, end_date=None):
    req = requests.get(SAMPLE_SCHEDULE_API)
    data_json = json.loads(req.text)["data"]
    data = json.loads(data_json)
    return pd.DataFrame(data)

Do note that ast.literal_eval is for Python code, whereas json.loads is for JSON that closely follows JavaScript code; the differences are for example true , false and null vs True, False and None. The former are the javascript syntax as used in JSON (and thus you would need json.loads; the latter is Python code, for which you would use ast.literal_eval.

Upvotes: 3

Ulrich Eckhardt
Ulrich Eckhardt

Reputation: 17415

The field "data" is a string, not a list. The content of that string seems to be JSON, too, so you have JSON encapsulated in JSON for some reason. If you can, fix that so that you only encode as JSON once. If that doesn't work, you can retrieve that field and decode it separately.

Upvotes: 0

Related Questions