Writing defaultdict(list) to file

Question

Previously asked a question Using defaultdict to parse multi delimiter file

While I do get the desired output based on the code, I am struggling to write it to a file as a table in this form

         count pos _pos _neg
31022550     
31022550    
31022550    
31022550

ids:

for key, rows in ids.iteritems():
     for row in rows:
         print '{}\t{}'.format(key, row)

31022550    {'count': '0', 'base': '=', 'pos': '20', '_neg': '0', '_pos': '0'}
31022550    {'count': '2', 'base': 'A', 'pos': '20', '_neg': '0', '_pos': '2'}
31022550    {'count': '0', 'base': 'C', 'pos': '20', '_neg': '0', '_pos': '0'}
31022550    {'count': '1391', 'base': 'G', 'pos': '20', '_neg': '672', '_pos': '719'}
31022550    {'count': '1', 'base': 'T', 'pos': '20', '_neg': '1', '_pos': '0'}
31022440    {'count': '0', 'base': 'N', 'pos': '20', '_neg': '0', '_pos': '0'}
31022550    {'count': '2', 'base': '+A', 'pos': '20', '_neg': '0', '_pos': '2'}
31022551    {'count': '0', 'base': '=', 'pos': '20', '_neg': '0', '_pos': '0'}
31022551    {'count': '960', 'base': 'A', 'pos': '20', '_neg': '464', '_pos': '496'}
31022551    {'count': '0', 'base': 'C', 'pos': '20', '_neg': '0', '_pos': '0'}
31022551    {'count': '13', 'base': 'G', 'pos': '20', '_neg': '9', '_pos': '4'}
31022551    {'count': '0', 'base': 'T', 'pos': '20', '_neg': '0', '_pos': '0'}
31022551    {'count': '0', 'base': 'N', 'pos': '20', '_neg': '0', '_pos': '0'}
31022551    {'count': '288', 'base': '+G', 'pos': '20', '_neg': '117', '_pos': '171'}
31022551    {'count': '9', 'base': '+GG', 'pos': '20', '_neg': '4', '_pos': '5'}
31022551    {'count': '1', 'base': '+GGG', 'pos': '20', '_neg': '0', '_pos': '1'}

Code

with open('mycsvfile.csv', 'w') as f:
    writer = csv.writer(f)
    for k, v in ids.iteritems():
        writer.writerow([k] + v)

Answer 1

Please find my answer below:

#Writing an instance of  Defaultdict(list) into a file is simpler than I 
# thought. Polymorphisms can solve this problem. 
# Create an object (dd) of type Defaultdict(list) (mydd) which is a subtype of
# dict, Assign mydd to dd and save dd into a file as json.

from collections import defaultdict
import json
dd = defaultdict(list)
dd["horse"].append("Mammal")
dd["Eagle"].append("Bird")
dd["Python"].append("Reptile")
print(dd)

# Let save this defaultdict instance in a file
mydd = dict
mydd = dd
print("#" * 70)
print(f"printing mydd ----:{mydd} ---- an instance of a dict")
# Now saving dd under mydd, an instance of a dict.
print("#" * 70)
with open("mydd.json", "w") as f:
    print("writing my defaultdict object into a file")
    f.write(json.dumps(mydd))
    f.close()
print("#" * 70)
    
mydd1 = dict
# opening the file and store it into an object from the file

with open("mydd.json", "r") as f:
    print("opening my defaultdict object into a file")
    mydd1 = json.load(f)
    print("printing the json file just saved above", mydd1)

OUTPUT IN TEXT

defaultdict(<class 'list'>, {'horse': ['Mammal'], 'Eagle': ['Bird'], 'Python': ['Reptile']})
######################################################################
printing mydd ----:defaultdict(<class 'list'>, {'horse': ['Mammal'], 'Eagle': ['Bird'], 'Python': ['Reptile']}) ---- an instance of a dict
######################################################################
writing my defaultdict object into a file
######################################################################
opening my defaultdict object into a file
printing the json file just saved above {'horse': ['Mammal'], 'Eagle': ['Bird'], 'Python': ['Reptile']}

output of the code below

Answer 2

I would do this (python 2):

with open('mycsvfile.csv', 'wb') as f:  # binary is better, avoids blank lines in some python 2 versions
    writer = csv.writer(f,delimiter="\t")
    keys=["count","pos","_pos","_neg"]
    writer.writerow([""]+keys)
    for k, vl in ids.iteritems():
        for v in vl:
            writer.writerow([k] + [v[key] for key in keys])

you need a double loop to iterate on the lists for each key. I have stored the column names in a list, so I can reuse it to build the rows in a list comprehension & for the title as well (first item doesn't have a title, I just left it blank)

now it looks like this:

        count   pos     _pos    _neg
31022550        0       20      0       0
31022550        2       20      2       0
31022550        0       20      0       0

(slightly shifted because tab character isn't wide enough, but not an issue to read it back)

Python 3 users would have to change:

with open('mycsvfile.csv', 'wb') as f:

by

with open('mycsvfile.csv', 'w',newline="") as f:

and

for k, vl in ids.iteritems():

by

for k, vl in ids.items():  # also works in python 2

note that the writerow double loop could be replaced by a single line, a double-loop, flat generator comprehension passed to writerows, faster to execute:

writer.writerows([k] + [v[key] for key in keys] for k, vl in ids.items() for v in vl)