CODEWITHSUNDEEP
pythonpandasdataframefor-loopnested-loops
Marc Schwambach
Marc Schwambach

Reputation: 438

Avoiding multiples nested for-loops

Pardon if I couldn't synthesize precisely my problem with my title, however I guess that by explaning it, things will get more clear.

My problem is that I have to perform a calculation utilizing every combination of a set of arrays and then, store the corresponding result in an array for each loop, as well as the arguments utilized for the calculation. Subsequently, I will store the NumPy arrays and the corresponding result of the calculation in a pandas dataframe

I am trying to avoid for loops as much as possible and I am not sure if there are other means to achieve my goal utilizing pandas or python methods which I am still not aware of. The problem that I am trying to solve is naturally more complex and would involve several more arrays as well as more complex data. So, to sum up, my questions is if there are other smart ways to avoid what I have been doing.

The code that I have been working on - written in a very matlab-ish way - has a similar logic to the following(this is just an illustrative simpler example):

max_x = 5
min_x = 1
x_1 = np.linspace(min_x, max_x, 5)
x_2 = np.linspace(min_x, max_x, 5)
x_3 = np.linspace(min_x, max_x, 5)
x_4 = np.linspace(min_x, max_x, 5)

x_result = np.array([])
x1 = np.array([])
x2 = np.array([])
x3 = np.array([])
x4 = np.array([])

for i in range(0,len(x_1)):
    for j in range(0,len(x_2)):
        for k in range(0,len(x_3)):
            for l in range(0, len(x_4)):
                        x_set = [x_1[i],x_2[j],x_3[k], x_4[l]]
                        x_calc = calculation_1(arg = x_set)                             
                        x1 = np.append(x1, x_1[i])
                        x2 = np.append(x2, x_2[j])
                        x3 = np.append(x3, x_3[k])
                        x4 = np.append(x4, x_4[l])
                        x_result = np.append(x_result, x_calc)

df_x = pd.DataFrame(np.array([x1, x2, x3, x4, x_result])).T

Upvotes: 0

Views: 305

Answers (2)

willeM_ Van Onsem
willeM_ Van Onsem

Reputation: 476813

If I understand it correctly, you want to implement some sort of cartesian product with arrays. We can do this by using np.meshgrid, like:

def cartesian_product(*arrs):
    return np.transpose(np.meshgrid(*arrs)).reshape(-1, len(arrs))

For example:

>>> x_1 = [1,2,3]
>>> x_2 = [3,4]
>>> x_3 = [5]
>>> cartesian_product(x_1, x_2, x_3)
array([[1, 3, 5],
       [1, 4, 5],
       [2, 3, 5],
       [2, 4, 5],
       [3, 3, 5],
       [3, 4, 5]])

You can then channel the items of this cross product through calculation_1 , for example with np.apply_along_axis(..):

np.apply_axis(calculation_1, 1, c)

We can then add that result as a new column, for example with sum:

>>> c = cartesian_product(x_1, x_2, x_3)
>>> np.hstack((c, np.apply_axis(sum, 1, c)[:, None]))
array([[ 1,  3,  5,  9],
       [ 1,  4,  5, 10],
       [ 2,  3,  5, 10],
       [ 2,  4,  5, 11],
       [ 3,  3,  5, 11],
       [ 3,  4,  5, 12]])

Upvotes: 3

maxwell
maxwell

Reputation: 855

You can rewrite n nested loops with recursion for example.

(And eventually you can write any recursion with single loop using queues)

Read this for details

Upvotes: -1

Related Questions