CODEWITHSUNDEEP
pythonjsoncsv
Eric Shreve
Eric Shreve

Reputation: 21

Converting & Formating JSON to CSV Correctly?

I have a JSON URL I want to convert to a CSV file. How can I do this with Python?

I tried:

import requests
import csv

params = {
  "format": "csv"
  }
r = requests.get('https://myaccount.srpnet.com/myaccountapi/api/outages/getall', params=params)
print(r.text)

# This bit of code will write the result of the query to output.csv

with open('C:/test/filename.csv', 'w+') as f:
    fieldnames = ["latitude"], ["longitude"]
    f.write(r.text)

The output .csv looks like this:

"[{""latitude"":33.6031326225"  longitude:-112.193951324    numberCustomersAffected:1   "outageProblem:""An underground power cable has failed. SRP crews are working to restore power."""  isMaintenanceOutage:false   estimatedUsersImpacted:0    "crossRoadText:""N 67TH AVE to N 59TH AVE and W THUNDERBIRD RD to W CACTUS RD"""    "estimatedRestorationTime:""2019-12-16T21:35:00Z""" "outageBegan:""2019-12-16T15:33:27Z"""  "outageId:""486858""}"  "{""latitude"":33.279829655"    longitude:-111.772819298    numberCustomersAffected:1   "outageProblem:""An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible."""  isMaintenanceOutage:true    estimatedUsersImpacted:0    "crossRoadText:""S GILBERT RD to S QUARTZ ST and E PECOS RD to E GERMANN RD"""  "estimatedRestorationTime:""2019-12-16T23:00:00Z""" "outageBegan:""2019-12-16T14:59:11Z"""  "outageId:""486875""}"  "{""latitude"":33.4974939093"   longitude:-111.984698683    numberCustomersAffected:16  "outageProblem:""An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible."""  isMaintenanceOutage:true    estimatedUsersImpacted:0    "crossRoadText:""N 44TH ST to N 48TH ST and E CAMPBELL AVE to E INDIAN SCHOOL RD""" "estimatedRestorationTime:""2019-12-16T21:50:00Z""" "outageBegan:""2019-12-16T15:48:35Z"""  "outageId:""486882""}"  "{""latitude"":33.377234657"    longitude:-111.644471686    numberCustomersAffected:11  "outageProblem:""We continue to investigate the cause."""   isMaintenanceOutage:false   estimatedUsersImpacted:0    "crossRoadText:""S HAWES RD to S ELLSWORTH RD and E BASELINE RD to E STEVEN POLLARD AVE"""  "estimatedRestorationTime:""2019-12-16T20:40:00Z""" "outageBegan:""2019-12-16T18:35:56Z"""  "outageId:""486922""}"  "{""latitude"":33.1435118621"   longitude:-111.521769468    numberCustomersAffected:12  "outageProblem:""An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible."""  isMaintenanceOutage:true    estimatedUsersImpacted:0    "crossRoadText:""N TOURMALINE DR to N SIERRA VISTA RD and E JUDD RD to W SILVERDALE RD"""   "estimatedRestorationTime:""2019-12-16T20:50:00Z""" "outageBegan:""2019-12-16T18:49:03Z"""  "outageId:""486924""}]"

What I want it to look like this:

latitude    longitude   numberCustomersAffected outageProblem   isMaintenanceOutage estimatedUsersImpacted  crossRoadText   estimatedRestorationTime    outageBegan outageId
33.60313262 -112.1939513    1   An underground power cable has failed. SRP crews are working to restore power.  FALSE   0   N 67TH AVE to N 59TH AVE and W THUNDERBIRD RD to W CACTUS RD    2019-12-16T21:35:00Z    2019-12-16T15:33:27Z    486858
33.50462573 -111.9944968    7   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   N 36TH ST to N 44TH ST and E CAMELBACK RD to E CAMPBELL AVE 2019-12-16T19:45:00Z    2019-12-16T14:41:08Z    486872
33.41878816 -111.6283175    2   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   N ELLSWORTH RD to N 96TH ST and E UNIVERSITY DR to E APACHE TRL 2019-12-16T16:45:00Z    2019-12-16T14:44:38Z    486873
33.51363707 -112.0268558    6   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   N 24TH ST to N 28TH ST and E MISSOURI AVE to E CAMELBACK RD 2019-12-16T16:55:00Z    2019-12-16T14:53:49Z    486874
33.27982966 -111.7728193    1   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   S GILBERT RD to S QUARTZ ST and E PECOS RD to E GERMANN RD  2019-12-16T23:00:00Z    2019-12-16T14:59:11Z    486875
33.44074458 -112.1861484    0   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   S 63RD AVE to S 55TH AVE and W VAN BUREN ST to W BUCKEYE RD 2019-12-16T18:05:00Z    2019-12-16T15:02:40Z    486876
33.41773662 -111.9926656    0   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   S 40TH ST to S 43RD WAY and E UNIVERSITY DR to E ELWOOD ST  2019-12-16T19:30:00Z    2019-12-16T15:26:58Z    486879
33.33877131 -111.7891725    2   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   S 132ND ST to S LINDSAY RD and W HACKAMORE AVE to E WARNER RD   2019-12-16T16:40:00Z    2019-12-16T15:35:35Z    486880
33.49749391 -111.9846987    16  An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   N 44TH ST to N 48TH ST and E CAMPBELL AVE to E INDIAN SCHOOL RD 2019-12-16T21:50:00Z    2019-12-16T15:48:35Z    486882
33.40325351 -111.8665199    12  An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   S DOBSON RD to S ALMA SCHOOL RD and W BROADWAY RD to W 8TH AVE  2019-12-16T20:45:00Z    2019-12-16T15:41:30Z    486883
33.5133441  -112.1491916    11  An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   GRAND AVE to N 39TH AVE and W MISSOURI AVE to W CAMELBACK RD    2019-12-16T18:45:00Z    2019-12-16T15:44:40Z    486884
33.50472637 -111.9348527    14  An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   N INVERGORDON RD to N SCOTTSDALE RD and E CHAPARRAL RD to E CAMELBACK RD    2019-12-16T17:50:00Z    2019-12-16T15:49:44Z    486886
33.51471289 -112.2795876    7   An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.  TRUE    0   N 107TH AVE to N 99TH AVE and W BETHANY HOME RD to W CAMELBACK RD   2019-12-16T18:55:00Z    2019-12-16T15:52:04Z    486887

The JSON API Request looks as follows: 

[{"latitude":33.6031326225,"longitude":-112.193951324,"numberCustomersAffected":1,"outageProblem":"An underground power cable has failed. SRP crews are working to restore power.","isMaintenanceOutage":false,"estimatedUsersImpacted":0,"crossRoadText":"N 67TH AVE to N 59TH AVE and W THUNDERBIRD RD to W CACTUS RD","estimatedRestorationTime":"2019-12-16T21:35:00Z","outageBegan":"2019-12-16T15:33:27Z","outageId":"486858"},{"latitude":33.279829655,"longitude":-111.772819298,"numberCustomersAffected":1,"outageProblem":"An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.","isMaintenanceOutage":true,"estimatedUsersImpacted":0,"crossRoadText":"S GILBERT RD to S QUARTZ ST and E PECOS RD to E GERMANN RD","estimatedRestorationTime":"2019-12-16T23:00:00Z","outageBegan":"2019-12-16T14:59:11Z","outageId":"486875"},{"latitude":33.4974939093,"longitude":-111.984698683,"numberCustomersAffected":16,"outageProblem":"An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.","isMaintenanceOutage":true,"estimatedUsersImpacted":0,"crossRoadText":"N 44TH ST to N 48TH ST and E CAMPBELL AVE to E INDIAN SCHOOL RD","estimatedRestorationTime":"2019-12-16T21:50:00Z","outageBegan":"2019-12-16T15:48:35Z","outageId":"486882"},{"latitude":33.377234657,"longitude":-111.644471686,"numberCustomersAffected":11,"outageProblem":"We are investigating the cause of the outage.","isMaintenanceOutage":false,"estimatedUsersImpacted":0,"crossRoadText":"S HAWES RD to S ELLSWORTH RD and E BASELINE RD to E STEVEN POLLARD AVE","estimatedRestorationTime":"2019-12-16T20:40:00Z","outageBegan":"2019-12-16T18:35:56Z","outageId":"486922"},{"latitude":33.1435118621,"longitude":-111.521769468,"numberCustomersAffected":12,"outageProblem":"An SRP maintenance crew is performing critical maintenance work to repair or upgrade equipment. Power will be restored as quickly as possible.","isMaintenanceOutage":true,"estimatedUsersImpacted":0,"crossRoadText":"N TOURMALINE DR to N SIERRA VISTA RD and E JUDD RD to W SILVERDALE RD","estimatedRestorationTime":"2019-12-16T20:50:00Z","outageBegan":"2019-12-16T18:49:03Z","outageId":"486924"}]

Upvotes: 1

Views: 87

Answers (3)

Hamall
Hamall

Reputation: 293

Try with pandas as following:

import requests
import json
import pandas

r = requests.get('https://myaccount.srpnet.com/myaccountapi/api/outages/getall')
data = r.json()
pandas.read_json(json.dumps(data)).to_csv('pandas.csv')

Upvotes: 3

mechanical_meat
mechanical_meat

Reputation: 169284

You need to write out the file using the csv module. In particular you need a DictWriter instance.

In addition it appears you're getting JSON back from the server. So you need the json module to parse that.

Below is a tested code sample.

import json, requests

params = {
  "format": "csv"
  }
r = requests.get('https://myaccount.srpnet.com/myaccountapi/api/outages/getall', 
                 params=params)

output = json.loads(r.text)
fieldnames = output[0].keys()

with open('C:/test/filename.csv', 'w+') as f:
    dw = csv.DictWriter(f,fieldnames=fieldnames)
    dw.writeheader()
    dw.writerows(output)

Upvotes: 0

sciroccorics
sciroccorics

Reputation: 2427

According to your incorrect formated CSV, I guess that r.text contains a list of dicts. So you have first to write the header row, then loop over all rows, and write the data row-by-row. Try something like this:

with open('C:/test/filename.csv', 'w+') as f:

    fieldnames = ["latitude", "longitude"]
    writer = csv.DictWriter(f, fieldnames=fieldnames)    
    writer.writeheader()

    for row in r.text:
        writer.writerow(row)

Upvotes: 0

Related Questions