CODEWITHSUNDEEP
pythonjson
user4372
user4372

Reputation: 23

Invalid JSON Data From requests.get

I am querying a whole-house power monitor (Neurio) that returns data in JSON format.

When I enter the URL in a Chrome browser, http://192.168.1.87/current-sample, I get properly formatted JSON data as follows:

{"sensorId":"0x0000C47F51019B7D","timestamp":"2016-12-24T14:56:08Z","channels":[{"type":"PHASE_A_CONSUMPTION","ch":1,"eImp_Ws":55552784178,"eExp_Ws":23,"p_W":3188,"q_VAR":321,"v_V":121.753},{"type":"PHASE_B_CONSUMPTION","ch":2,"eImp_Ws":62493402411,"eExp_Ws":23,"p_W":3499,"q_VAR":263,"v_V":120.334},{"type":"CONSUMPTION","ch":3,"eImp_Ws":118046186640,"eExp_Ws":41,"p_W":6687,"q_VAR":584,"v_V":121.044}],"cts":[{"ct":1,"p_W":3188,"q_VAR":321,"v_V":121.753},{"ct":2,"p_W":3499,"q_VAR":263,"v_V":120.334},{"ct":3,"p_W":0,"q_VAR":0,"v_V":0.000},{"ct":4,"p_W":0,"q_VAR":0,"v_V":121.747}]}

which parses correctly in JSONLint.

When I attempt to pull the same data in Python using the following line of code: 

pvdata = requests.get('http://'+neurioip+'/current-sample').json()

The returned data includes invalid characters as in the following example. (Data retrieved by simply printing pvdata)

{u'channels': [{u'eExp_Ws': 23, u'v_V': 122.434, u'ch': 1, u'eImp_Ws': 55554346060, u'q_VAR': 305, u'p_W': 1489, u'type': u'PHASE_A_CONSUMPTION'}, {u'eExp_Ws': 23, u'v_V': 120.981, u'ch': 2, u'eImp_Ws': 62495160471, u'q_VAR': 237, u'p_W': 1872, u'type': u'PHASE_B_CONSUMPTION'}, {u'eExp_Ws': 41, u'v_V': 121.708, u'ch': 3, u'eImp_Ws': 118049506582, u'q_VAR': 542, u'p_W': 3360, u'type': u'CONSUMPTION'}], u'sensorId': u'0x0000C47F51019B7D', u'cts': [{u'p_W': 1489, u'q_VAR': 305, u'v_V': 122.434, u'ct': 1}, {u'p_W': 1872, u'q_VAR': 237, u'v_V': 120.981, u'ct': 2}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 0.0, u'ct': 3}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 122.432, u'ct': 4}], u'timestamp': u'2016-12-24T15:03:42Z'}

Data retreived using the following code:

for keys,values in pvdata.items(): print(keys) print(values)

channels [{u'eExp_Ws': 23, u'v_V': 122.843, u'ch': 1, u'eImp_Ws': 55555370977, u'q_VAR': 14, u'p_W': 230, u'type': u'PHASE_A_CONSUMPTION'}, {u'eExp_Ws': 23, u'v_V': 121.088, u'ch': 2, u'eImp_Ws': 62496733790, u'q_VAR': -3, u'p_W': 661, u'type': u'PHASE_B_CONSUMPTION'}, {u'eExp_Ws': 41, u'v_V': 121.965, u'ch': 3, u'eImp_Ws': 118052104817, u'q_VAR': 12, u'p_W': 890, u'type': u'CONSUMPTION'}] sensorId 0x0000C47F51019B7D cts [{u'p_W': 230, u'q_VAR': 14, u'v_V': 122.843, u'ct': 1}, {u'p_W': 661, u'q_VAR': -3, u'v_V': 121.088, u'ct': 2}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 0.0, u'ct': 3}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 122.84, u'ct': 4}] timestamp 2016-12-24T15:25:16Z

The "u" characters makes this unparsable in JSONlint or in subsequent lines of code in my program.

I've looked at default character encoding in my Python environment but that doesn't seem to lead anywhere. I'm looking for other ideas to investigate.

Upvotes: 2

Views: 1299

Answers (2)

Alex Hall
Alex Hall

Reputation: 36013

.json() is what parses the JSON. It's done. You have a python dictionary which you can use normally now (as you've already seen by iterating through it).

Upvotes: 4

fuglede
fuglede

Reputation: 18201

It is unparsable since the string representation of a dict simply isn't JSON; you need to remarshall the dict to JSON if that's what you need:

In [36]: d = {u'channels': [{u'eExp_Ws': 23, u'v_V': 122.434, u'ch': 1, u'eImp_Ws': 55554346060, u'q_VAR': 305, u'p_W': 1489, u'type': u'PHASE_A_CONSUMPTION'}, {u'eExp_Ws': 23, u'v_V': 120.981, u'ch': 2, u'eImp_Ws': 62495160471, u'q_VAR': 237, u'p_W': 1872, u'type': u'PHASE_B_CONSUMPTION'}, {u'eExp_Ws': 41, u'v_V': 121.708, u'ch': 3, u'eImp_Ws': 118049506582, u'q_VAR': 542, u'p_W': 3360, u'type': u'CONSUMPTION'}], u'sensorId': u'0x0000C47F51019B7D', u'cts': [{u'p_W': 1489, u'q_VAR': 305, u'v_V': 122.434, u'ct': 1}, {u'p_W': 1872, u'q_VAR': 237, u'v_V': 120.981, u'ct': 2}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 0.0, u'ct': 3}, {u'p_W': 0, u'q_VAR': 0, u'v_V': 122.432, u'ct': 4}], u'timestamp': u'2016-12-24T15:03:42Z'}

In [38]: json.dumps(d)
Out[38]: '{"sensorId": "0x0000C47F51019B7D", "timestamp": "2016-12-24T15:03:42Z", "cts": [{"q_VAR": 305, "ct": 1, "v_V": 122.434, "p_W": 1489}, {"q_VAR": 237, "ct": 2, "v_V": 120.981, "p_W": 1872}, {"q_VAR": 0, "ct": 3, "v_V": 0.0, "p_W": 0}, {"q_VAR": 0, "ct": 4, "v_V": 122.432, "p_W": 0}], "channels": [{"q_VAR": 305, "type": "PHASE_A_CONSUMPTION", "p_W": 1489, "v_V": 122.434, "ch": 1, "eImp_Ws": 55554346060, "eExp_Ws": 23}, {"q_VAR": 237, "type": "PHASE_B_CONSUMPTION", "p_W":
1872, "v_V": 120.981, "ch": 2, "eImp_Ws": 62495160471, "eExp_Ws": 23}, {"q_VAR": 542, "type": "CONSUMPTION", "p_W": 3360, "v_V": 121.708, "ch": 3, "eImp_Ws": 118049506582, "eExp_Ws": 41}]}'

Upvotes: 0

Related Questions