Python: iterating and modifying nested lists

Question

I apologize if this question has been asked before, as it seems to be very basic. Unfortunately, when I searched for my question, I could only find other questions asking how to iterate over a list of lists, and none of these questions touched on the specific behavior I am asking about.

I am aware that in python, when you equate two lists, you are not actually copying that list in memory, just creating a new alais pointing to that list in memory. so something like

listA = [1,2,3]
listB = listA
listB[0] = 5
print(listA) #prints [5,2,3]

makes perfect sense to me.

I also know that you can modify mutable types (like lists) in a for loop, while for other types (like integers), you cannot, and must modify the original list. for example

listA = [1,2,3]
listB = [4,5,6]
for Int in listA:
    Int +=1
print(listA) #doesn't work, prints [1,2,3]

for List in [listA,listB]:
    List[2] = 100
print(listA) #works, prints [1,2,100]

my problem appeared when I tried to combine these two principles. Here is an example of what I tried to do:

x = [1.2345,0.543895,0.0]
y = [2,3,4]
z = [65.34,3.248578493,1.11111]
for coord in [x,y,z]:
    rounded_coord = [round(item,3) for item in coord]
    coord = rounded_coord
print(x,y,z) #prints unmodified numbers

In this example, 'coord' is a list, and therefore I should be able to modify it, just like listA in my previous examples, but I can't. I have to use enumerate:

x = [1.2345,0.543895,0.0]
y = [2,3,4]
z = [65.34,3.248578493,1.11111]
coordlist = [x,y,z]
for idx,coord in enumerate(coordlist):
    coordlist[idx] = [round(item,3) for item in coord]
print(coordlist)

Why doesn't my original attempt work?

Answer 1

Your first example did not work because coord is only a reference to to the lists x, y, and z. When you reassign coord, you are overwriting the original reference and setting coord to reference a new list object.

In other words coord is not a pointer to each list. Rather, it is only a reference. And when you reassign coord, you lose the original reference.

You can remedy this problem by mutating the list object coord is referencing:

>>> x = [1.2345, 0.543895, 0.0]
>>> y = [2, 3, 4]
>>> z = [65.34, 3.248578493, 1.11111]
>>> 
>>> for coord in [x, y, z]:
    coord[:] = [round(item,3) for item in coord]


>>> x
[1.234, 0.544, 0.0]
>>> y
[2, 3, 4]
>>> z
[65.34, 3.249, 1.111]
>>>

Answer 2

The issue is that assignment never mutates. It merely creates a new reference, or reassigns a reference (which is what you are doing). You have to use a mutator method, so for example:

In [1]: x = [1.2345,0.543895,0.0]
   ...: y = [2,3,4]
   ...: z = [65.34,3.248578493,1.11111]
   ...: for coord in [x,y,z]:
   ...:     coord[:] = [round(item,3) for item in coord]
   ...: print(x,y,z)
   ...:
[1.234, 0.544, 0.0] [2, 3, 4] [65.34, 3.249, 1.111]

So, even though coord[:] = [round(item,r) for item in cord] looks like a regular assignment, it is not. It is syntactic sugar for calling a mutator method, __setitem__ to be precise:

In [2]: coord
Out[2]: [65.34, 3.249, 1.111]

In [3]: coord.__setitem__(slice(None,None), ['foo','bar','baz'])

In [4]: coord
Out[4]: ['foo', 'bar', 'baz']

Similarly, List[0] = 100 is not assignment, it is syntactic sugar for:

In [6]: List
Out[6]: [1, 2, 3]

In [7]: List[0] = 99

In [8]: List
Out[8]: [99, 2, 3]

In [9]: List.__setitem__(0, 'foo')

In [10]: List
Out[10]: ['foo', 2, 3]

You can't modify immutable types because, by definition, they have no mutator methods, even though there are operations that look like mutator methods:

x = 10
x += 10 # looks like a mutator method but actually assigns a new object to x

Note, a str is similar to a list. You can slice it, but the corresponding mutator method doesn't exist:

In [17]: s = 'abcdefg'

In [18]: s[:3]
Out[18]: 'abc'

In [19]: s.__getitem__(slice(0,3))
Out[19]: 'abc'

In [20]: s.__setitem__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-20-44ee26769121> in <module>()
----> 1 s.__setitem__

AttributeError: 'str' object has no attribute '__setitem__'

Answer 3

Every time you use a loop of the form:

for item in list:
    #do something

The item becomes a reference to each element in the list as it loops through. Using modifiers such as:

item[2] = 5

will work because you are modifying the reference directly.

However, when you try to set the reference equal to another variable:

item = some_other_thing

you are not actually modifying the item, you are modifying the reference to the item, and have therefor never changed the original item, just lost your reference to it.

Answer 4

To modify elements of the list, you need to provide the index and then modify. In other words, this will not modify anything in the list:

arr=[1,2,3]
for x in arr:
     x+=1 #not modifying the values in the list
     print(x) #this will print 2,3,4
print(arr) #prints [1,2,3]

So even though you incremented the values but you did not replace the values in the list

What you need to do is, use the index to change the values

arr=[1,2,3]
for x in range(len(arr)): #len(arr) evaluates to 3 and range(3) makes an array of numbers from 0 up to but not including 3
   arr[x]=arr[x]+1
print[arr] #Now prints [2,3,4]

Answer 5

In all your cases, the loop variable is just an alias for each element in the thing being looped over.

The reason List in your second example works like you want it to, but coord in your third example doesn't, is that in the second example, you're changing an element of the list pointed to by List, and therefore actually updating the underlying data.

In the third example, you're trying to directly change coord, and thus are just updating the alias to point to a different list, instead of using the alias to access the underlying array and changing it. As a result, coord now just points to a different list. Consider this modified version of your second example:

for List in [listA,listB]:
    List = [7,8,9]
print(listA) # prints [1,2,3]; underlying data not changed
print(List) # prints [7,8,9]; List now points to the new array

Answer 6

In your example:

x = [1.2345,0.543895,0.0]
y = [2,3,4]
z = [65.34,3.248578493,1.11111]
for coord in [x,y,z]:
    rounded_coord = [round(item,3) for item in coord]
    coord = rounded_coord
print(x,y,z) #prints unmodified numbers

coord is not a list - it's a pointer to a list in your ad-hoc created [x, y, z] list. Since this is a relatively simple example, here is how this unpacks without a loop:

coord = x
rounded_coord = [round(item,3) for item in coord]
coord = rounded_coord

coord = y
rounded_coord = [round(item,3) for item in coord]
coord = rounded_coord

coord = z
rounded_coord = [round(item,3) for item in coord]
coord = rounded_coord

See a problem with it?

If you really wanted to change the element itself, you'd either have to reference it by index in its container list on replacement (as you've noticed that it works with enumerate) or to replace the value in place:

for coord in [x,y,z]:
    coord[:] = [round(item,3) for item in coord]  # assuming not overriden __setslice__()

Answer 7

'coord' is a list, and therefore I should be able to modify it...

Almost, but not quite. coord is a variable that stores a reference to the each of the original lists in turn per iteration.

rounded_coord is also a variable that stores a reference to a new list.

Now, doing coord = rounded_coord will make the variable coord point to the same reference as rounded_coord. Meaning, the original contents of coords will remain unchanged while the reference that coord points to changes.

Example:

>>> x = [1, 2, 3, 4, 5]
>>> for l in [x]:
...    print(id(l))
...    new_l = [1]
...    l = new_l
...    print(id(l))
...    
4309421160
4309421592

By the way, id prints a 10 digit number representing the reference a variable points to. You can see that at the start vs at the end, the reference stored in l changes.