CODEWITHSUNDEEP
pythonjsonpython-3.x
TrentWoodbury
TrentWoodbury

Reputation: 911

How to create newline delimited json-lines file from list of python dictionaries

I am trying to create a JSON-lines file of data so that is compatible with google cloud AI platform's requirements for online prediction.

Right now I have a list of dictionaries for each of my data points. It looks like this:

data = [{'values': [0,1,0], 'key': 0}, {'values': [1,1,0], 'key': 1}]

I'm exporting this data to data.json with the following code:

import json
json_filepath = "data.json"
with open(json_filepath, 'w') as f:
    json.dump(data, f)

The problem is, this data.json file then looks exactly like my data (viz. a list of dictionaries). How can I make this data.json file a new-line delimited collection of each dictionary in the list? In other words, how can I make it look like this:

{'values': [0,1,0], 'key': 0}
{'values': [1,1,0], 'key': 1}

Upvotes: 10

Views: 12568

Answers (2)

Djaouad
Djaouad

Reputation: 22766

You can loop through the array and dump each object followed by a new line '\n':

with open(json_filepath, 'w') as f:
    for d in data:
        json.dump(d, f)
        f.write('\n')

Alternatively, you can use a one-liner using json.dumps, str.join, and map:

with open(json_filepath, 'w') as f:
    f.write('\n'.join(map(json.dumps, data)))

Upvotes: 20

Andrew
Andrew

Reputation: 151

with open(json_filepath, "w") as f:
    for datum in data:
        f.write(json.dumps(datum))
        f.write("\n")

I should also mention in general this is known as the "JSON Lines" data format.

Upvotes: 4

Related Questions