CODEWITHSUNDEEP
python
guilin 桂林
guilin 桂林

Reputation: 17422

python RuntimeError: dictionary changed size during iteration

I have obj like this

{hello: 'world', "foo.0.bar": v1, "foo.0.name": v2, "foo.1.bar": v3}

It should be expand to 

{ hello: 'world', foo: [{'bar': v1, 'name': v2}, {bar: v3}]}

I wrote code below, splite by '.', remove old key, append new key if contains '.', but it said RuntimeError: dictionary changed size during iteration

def expand(obj):
    for k in obj.keys():
        expandField(obj, k, v)

def expandField(obj, f, v):
    parts = f.split('.')
    if(len(parts) == 1):
        return
    del obj[f]
    for i in xrange(0, len(parts) - 1):
        f = parts[i]
        currobj = obj.get(f)
        if (currobj == None):
            nextf = parts[i + 1]
            currobj = obj[f] = re.match(r'\d+', nextf) and [] or {}
        obj = currobj
    obj[len(parts) - 1] = v

for k, v in obj.iteritems():

RuntimeError: dictionary changed size during iteration

Upvotes: 21

Views: 70963

Answers (5)

Damien
Damien

Reputation: 1724

You might want to copy your keys in a list and iterate over your dict using the latter, eg:

def expand(obj):
    keys = list(obj.keys())  # freeze keys iterator into a list
    for k in keys:
        expandField(obj, k, v)

I let you analyse if the resulting behavior suits your expected results.

Edited as per comments, thank you !

Upvotes: 16

Tomasz Bartkowiak
Tomasz Bartkowiak

Reputation: 14958

For those experiencing

RuntimeError: dictionary changed size during iteration

also make sure you're not iterating through a defaultdict when trying to access a non-existent key! I caught myself doing that inside the for loop, which caused the defaultdict to create a default value for this key, causing the aforementioned error.

The solution is to convert your defaultdict to dict before looping through it, i.e.

d = defaultdict(int)
d_new = dict(d)

or make sure you're not adding/removing any keys while iterating through it.

Upvotes: 4

mausamsion
mausamsion

Reputation: 230

Rewriting this part

def expand(obj):
    for k in obj.keys():
        expandField(obj, k, v)

to the following

def expand(obj):
    keys = obj.keys()
    for k in keys:
        if k in obj:
            expandField(obj, k, v)

shall make it work.

Upvotes: 0

xpros
xpros

Reputation: 2446

I had a similar issue with wanting to change the dictionary's structure (remove/add) dicts within other dicts.

For my situation I created a deepcopy of the dict. With a deepcopy of my dict, I was able to iterate through and remove keys as needed.Deepcopy - PythonDoc

A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

Hope this helps!

Upvotes: 3

Scott Hunter
Scott Hunter

Reputation: 49803

Like the message says: you changed the number of entries in obj inside of expandField() while in the middle of looping over this entries in expand.

You might try instead creating a new dictionary of the form you wish, or somehow recording the changes you want to make, and then making them AFTER the loop is done.

Upvotes: 30

Related Questions