CODEWITHSUNDEEP
pythonjsoncsv
andrew
andrew

Reputation: 389

Python CSV writer missing row of data

I'm experiencing a problem when I dump json data into a CSV file. There is typically a block of json data that is missing from my the CSV file, but can be seen if I print the json in the console or to a file.

Essentially I am calling a service twice and receiving back two json responses that I parse and dump into a CSV file. The service can only be called for 7 day increments (unix time), so I have implemented logic to call the service for this increment over a period of time.

I'm using the python vanilla json and csv libraries.

First the CSV is created with headers:

with open ('history_' + datetime.datetime.now().strftime("%Y-%m-%d_%H-%M-%S")+'.csv', 'wb') as outcsv:
    writer = csv.writer(outcsv)
    writer.writerow(["Column1","Column2", "Column3", "Column4", "Column5",
              "Column6"])

Then, I have a counter that calls the service twice, fifty times (following the open of the CSV file):

while y<50:
    jsoResponseOne = getJsonOne(7)
    jsonResponseTwo = getJsonTwo(7)

Example json response:

{"Value": 
  [
    {"ExampleName": "Test", 
     "ExampleNameTwo": "Test2", 
     "ExampleDate": "1436103790", 
     "ExampleCode": 00000001, 
     "ExampleofExample": "abcd", 
     "AnotherExample": "hello"},
     {"ExampleName": "Test2", 
     "ExampleNameTwo": "Test3", 
     "ExampleDate": "1436103790", 
     "ExampleCode": 00000011, 
     "ExampleofExample": "abcd", 
     "AnotherExample": "hello2"},
  ]
}

The CSV output columns would look like:

ExampleName   ExampleNameTwo   ExampleDate   ExampleCode  ExampleofExample   AnotherExample

Finally, the CSV is written as follows:

for item in jsonResponseOne['Value']:
                row = []
                row.append(str(item['ExampleName'].encode('utf-8')))
                if item.get("ExampleNameTwo"):
                    row.append(str(item["ExampleNameTwo"]))
                else:
                    row.append("None")
                row.append(str(item['ExampleDate']))
                row.append(str(item['ExampleCode'].encode('utf-8')))
                row.append(str(item['ExampleofExample'].encode('utf-8')))
                row.append(str(item['AnotherExample'].encode('utf-8')))
                writer.writerow(row)
for item in jsonResponseTwo['Value']:
                anotherRow= []
                anotherRow.append(str(item['ExampleName'].encode('utf-8')))
                if item.get("ExampleNameTwo"):
                    anotherRow.append(str(item["ExampleNameTwo"]))
                else:
                    anotherRow.append("None")
                anotherRow.append(str(item['ExampleDate']))
                anotherRow.append(str(item['ExampleCode'].encode('utf-8')))
                anotherRow.append(str(item['ExampleofExample'].encode('utf-8')))
                anotherRow.append(str(item['AnotherExample'].encode('utf-8')))
                writer.writerow(anotherRow)

Why could my CSV output be missing an entire row of data (a block of data from the JSON response)?

Upvotes: 0

Views: 914

Answers (1)

andrew
andrew

Reputation: 389

Resolved.

The Python script had an indentation issue in the one of the while loops, causing some data to be skipped over and not written to the CSV file.

Upvotes: 1

Related Questions