Adding multiple dictionaries to a key in python dictionary

Question

I am trying to add multiple dictionaries to a key. e.g.

value = { column1 : {entry1 : val1}
                    {entry2 : val2}

          column2 : {entry3 : val3}
                    {entry4 : val4}
}

What exactly I am trying to do with this code is: There is a file.txt which has columns and valid entries for that header. I am trying to make a dictionary with columns as key and for each column another dictionary for each valid entry. So I am parsing the text file line by line to find the pattern for column and entries and storing it in a variable, check if the column(which is a key) already exists in the dictionary, if exists then add another dictionary to the column, if not create a new entry. I Hope this makes sense.

Sample contents of file.txt

blah blah Column1 blah blah
    entry1 val1
    entry2 val2
blah blah Column2 blah blah
    entry3 val3
    entry4 val4

My code:

from __future__ import unicode_literals
import os, re, string, gzip, fnmatch, io
from array import *

header = re.compile(...) #some regex
valid_entries = re.compile(---) #some regex

matches=[]
entries=[]
value = {'MONTH OF INTERVIEW' : {'01': 'MIN VALUE'}}
counter = 0
name = ''

f =open(r'C:/file.txt')

def exists(data, name):
    for key in data.keys():
        if key == name :
            print "existing key : " + name
            return True
        else :
            return False
for line in f:

    col = ''
    ent = ''
    line = re.sub(ur'\u2013', '-', line)
    line = re.sub(ur'\u2026', '_', line)
    m = header.match(line)
    v = valid_entries.match(line)

    if m:
        name= ''
        matches.append(m.groups())
        _,_, name,_,_= m.groups()
        #print "name : " + name

    if v:
        entries.append(v.groups())
        ent,col= v.groups()
        #print v.groups()
        #print "col :" + col
        #print "ent :" + ent

    if (name is not None) and (ent is not None) and (col is not None):
        print value
        if exists(value, name):
            print 'inside existing loop'
            value[name].update({ent:col})
        else:
            value.update({name:{ent:col}})

print value

problem with this code is , it is replacing the values of the sub dictionary and also it is not adding all the values to the dictionary.

I am new to python, so this could be a naive approach to handle this kind of situation. If you think there is a better way of getting what I want, I would really appreciate if you tell me.

Answer 1

You can use defaultdict and regex for this (demo here):

with open('/path/to/file.txt', 'rU') as f:     # read the contents from the file
    lines = f.readlines()

import re
from collections import defaultdict
d = defaultdict(list)                          # dict with default value: []

lastKey = None
for line in lines:
    m = re.search('Column\d',line)             # search the current line for a key
    if m: lastKey = m.group()
    else:
        m = re.search('(?<=entry\d ).*',line)  # search the current line for a value
        if m: d[lastKey].append(m.group())     # append the value

Output:

[('Column1', ['val1', 'val2']), ('Column2', ['val3', 'val4'])]

---

Note: Of course, the above code assumes your file.txt was formatted as in your example. For your real file.txt data you might have to adjust the regex.

Answer 2

Dictionaries have only one value per key. The trick is to make that value a container too, like a list:

value = {
    'column1': [{entry1 : val1}, {entry2 : val2}]
    'column2': [{entry3 : val3}, {entry4 : val4}]
}

Use dict.setdefault() to insert a list value when there is no value yet:

if name is not None and ent is not None and col is not None:
    value.setdefault(name, []).append({ent: col})

You could just make the values one dictionary with multiple (ent, col) key-value pairs here:

if name is not None and ent is not None and col is not None:
    value.setdefault(name, {})[ent] = col

Your exists() function was overcomplicating a task dictionaries excel at; testing for a key is done using in instead:

if name in value:

would have sufficed.

Answer 3

I would keep the keys as a list of dictionaries, so you can extend or append

>>> d = {}
>>> d[1] = [{'a': 1}]
>>> d[1].append({'b':2})
>>> d
{1: [{'a': 1}, {'b': 2}]}