CODEWITHSUNDEEP
pythonjson
Vaishak Suresh
Vaishak Suresh

Reputation: 5845

Write multiple lists to a JSON file in python

Assume I have the following lists

list1 = [{"created_at": "2012-01-31T10:00:04Z"},{"created_at": "2013-01-31T10:00:04Z"}] 
list2 = [{"created_at": "2014-01-31T10:00:04Z"}]

I can write the first list to a JSON file using json.dump(list1,file,indent=2) and the result is

[
  {
    "created_at": "2012-01-31T10:00:04Z"
  },
  {
    "created_at": "2013-01-31T10:00:04Z"
  }
]

My question is, how do I append the contents of the second list? if I simple do json.dump(list2,file,indent=2), it results in an invalid JSON file as below.

[
  {
    "created_at": "2012-01-31T10:00:04Z"
  },
  {
    "created_at": "2013-01-31T10:00:04Z"
  }
][
  {
    "created_at": "2014-01-31T10:00:04Z"
  }
]

Edit: The lists are created dynamically by parsing about 8000 files. The above lists are just example. I could potentially be writing 8000 lists to the JSON file, so simple appending will not work.

Upvotes: 2

Views: 7867

Answers (4)

watbywbarif
watbywbarif

Reputation: 6977

I had similar problem, wanted valid json, not json lines, made custom solution:

# a+ is not good beacuse you can't seek before last end
open(out_path, 'r+' if path.isfile(out_path) else 'w+') as o:

# had multiple open calls, so little more complication

    for obj in data:
        for key in columns:
            with open_output(json_path, key) as o:
                o.seek(0, io.SEEK_END)
                if o.tell() == 0:
                    o.write('[\n')
                else:
                    o.seek(o.tell() - 3)
                    o.write(',\n')
                json.dump({key: obj[key] if key in obj else columns[key]}, o, indent=2)
                o.write('\n]')

Output file is valid json after each write, indentation was no relevant for me, but one line per object was not readable so ended up with this.

Upvotes: 0

Iqra.
Iqra.

Reputation: 763

I found a little lacking in the explanation given below that's why trying to make a point considered over here. A Json file can have a single parent element. Therefore, if at the first iteration, you dump 1st list then at the 2nd iteration, you will get the formatting error in the file. B/c Json demands these two lists to be wrapped inside one list/array before dumping.

Therefore, you store all lists in one list (either using appending or any other above-mentioned methods). And then you dump this aggregated list into your Json file. However, if you do not want to do so, you will have to create different files for your different lists.

Upvotes: 0

SmartElectron
SmartElectron

Reputation: 1441

When parse the files append (or extend) to a unique list and finally convert to JSON. Assume that your function for parse is parse.

>>> import json
>>> result = []
>>> for file in files:
...     result.append(parse(file))
...
>>> json.dump(result, file1, indent=2)

Upvotes: 1

user1907906
user1907906

Reputation: 

In [1]: import json

In [2]: list1 = [{"created_at": "2012-01-31T10:00:04Z"},{"created_at": "2013-01-31T10:00:04Z"}] 

In [3]: list2 = [{"created_at": "2014-01-31T10:00:04Z"}] 

In [4]: list1.extend(list2)

In [5]: json.dumps(list1)
Out[5]: '[{"created_at": "2012-01-31T10:00:04Z"}, {"created_at": "2013-01-31T10:00:04Z"}, {"created_at": "2014-01-31T10:00:04Z"}]'

or

In [8]: json.dumps(list1 + list2)
Out[8]: '[{"created_at": "2012-01-31T10:00:04Z"}, {"created_at": "2013-01-31T10:00:04Z"}, {"created_at": "2014-01-31T10:00:04Z"}]'

Upvotes: 3

Related Questions