CODEWITHSUNDEEP
pythonperformancelist
Amir
Amir

Reputation: 1087

Efficient way to add extra element to lists in Python

I have list of list like:

[[0, 1, 2], 
 [3, 1], 
 [2, 0, 1], 
 [4, 3], 
 [1, 3, 4]]

list values are between 0 and 5. Now I want to 5+(numbers below 5 that do not exist to each list). For example, In the first list we have 0,1,2 while we do not have 3, 4 so we want to add 5+3=8 and 5+4=9 so ultimately he new list should have 0,1,2,8,9. The final out put should be:

[[0, 1, 2, 8, 9],
 [3, 1, 5, 7, 9],
 [2, 0, 1, 8, 9],
 [4, 3, 5, 6, 7],
 [1, 3, 4, 5, 7]]

To do so I wrote the following python code: 

k = -1
for i in X1:
    k = k + 1
    for j in range(5, 10):        
        if j - 5 not in i:
            X[k].append(j)

However for large data set (list) this takes for ever. I am wondering if there is any more efficient way/suggestion I can use for this case?

Upvotes: 1

Views: 751

Answers (2)

Aaditya Ura
Aaditya Ura

Reputation: 12679

With one loop with One line solution:

Your job can be done using one loop only , Here are two method which can do your job in one loop only:

First method using map function :

[large[i].extend(map(lambda x: x + 5, list({0, 1, 2, 3, 4} - set(j)))) for i,j in enumerate(large)]

print(large)

output:

[[0, 1, 2, 8, 9], [3, 1, 5, 7, 9], [2, 0, 1, 8, 9], [4, 3, 5, 6, 7], [1, 3, 4, 5, 7]]

Second method using numpy:

import numpy as np


[large[i].extend(np.asarray(list({0, 1, 2, 3, 4} - set(j))) + 5) for i,j in enumerate(large)]

print(large)

output:

[[0, 1, 2, 8, 9], [3, 1, 5, 7, 9], [2, 0, 1, 8, 9], [4, 3, 5, 6, 7], [1, 3, 4, 5, 7]]

Upvotes: 1

Moses Koledoye
Moses Koledoye

Reputation: 78554

Running the in check (an O(n) operation for lists) for each j-5 within the nested loop (an O(nxm) operation) slows down the performance of your code - because lookup is slow and vanilla for loops in Python are not cheap.

You can simply take set differences and add 5, then extend the inner list with the result:

st = set(range(5)) 
for l in X1:
    l.extend(sorted([x+5 for x in st.difference(l)]))
print(X1)

You'll go much faster if you don't actually need to sort the new items you're appending, since the inner lists are not sorted anyway, as sorting throws in an extra O(nlogn) complexity.

Also the for loop in the list comprehension runs faster than a vanilla for loop, so I can argue the wall time performance of the nesting here is much better than the original.

The set difference out performs repeatedlist.__contains__ while the single list.extend also out performs repeated list.append.

[[0, 1, 2, 8, 9], 
 [3, 1, 5, 7, 9], 
 [2, 0, 1, 8, 9], 
 [4, 3, 5, 6, 7], 
 [1, 3, 4, 5, 7]]

Upvotes: 6

Related Questions