CODEWITHSUNDEEP
pythonnumpymatrixmultidimensional-array
Mahmoud Ayman
Mahmoud Ayman

Reputation: 197

Fill up a 2D array while iterating through it

An example what I want to do is instead of doing what is shown below:

Z_old = [[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]]
for each_axes in range(len(Z_old)):
    for each_point in range(len(Z_old[each_axes])):
        Z_old[len(Z_old)-1-each_axes][each_point] = arbitrary_function(each_point, each_axes)

I want now to not initialize the Z_old array with zeroes but rather fill it up with values while iterating through it which is going to be something like the written below although it's syntax is horribly wrong but that's what I want to reach in the end.

 Z = np.zeros((len(x_list), len(y_list))) for Z[len(x_list) -1 - counter_1][counter_2] is equal to power_at_each_point(counter_1, counter_2] for counter_1 in range(len(x_list)) and counter_2 in range(len(y_list))]

Upvotes: 0

Views: 134

Answers (2)

abarnert
abarnert

Reputation: 365737

As I explained in my answer to your previous question, you really need to vectorize arbitrary_function.

You can do this by just calling np.vectorize on the function, something like this:

Z = np.vectorize(arbitrary_function)(np.arange(3), np.arange(5).reshape(5, 1))

But that will only give you a small speedup. In your case, since arbitrary_function is doing a huge amount of work (including opening and parsing an Excel spreadsheet), it's unlikely to make enough difference to even notice, much less to solve your performance problem.

The whole point of using NumPy for speedups is to find the slow part of the code that operates on one value at a time, and replace it with something that operates on the whole array (or at least a whole row or column) at once. You can't do that by looking at the very outside loop, you need to look at the very inside loop. In other words, at arbitrary_function.

In your case, what you probably want to do is read the Excel spreadsheet into a global array, structured in such a way that each step in your process can be written as an array-wide operation on that array. Whether that means multiplying by a slice of the array, indexing the array using your input values as indices, or something completely different, it has to be something NumPy can do for you in C, or NumPy isn't going to help you.

If you can't figure out how to do that, you may want to consider not using NumPy, and instead compiling your inner loop with Cython, or running your code under PyPy. You'll still almost certainly need to move the "open and parse a whole Excel spreadsheet" outside of the inner loop, but at least you won't have to figure out how to rethink your problem in terms of vectorized operations, so it may be easier for you.

Upvotes: 2

Joran Beasley
Joran Beasley

Reputation: 113988

rows = 10
cols = 10
Z = numpy.array([ arbitrary_function(each_point, each_axes) for each_axes in range(cols) for each_point in range(rows) ]).reshape((rows,cols))

maybe?

Upvotes: 0

Related Questions