CODEWITHSUNDEEP
pythonarraysperformancenumpymultidimensional-array
Ethunxxx
Ethunxxx

Reputation: 1337

How to efficiently create a multidimensional numpy array with entries depending only on one dimension index?

How can the following numpy array be created more efficiently using built-in functions?

import numpy as np
MyArray = np.zeros([10,10,10,10,10])
for i in range(10):
    for j in range(10):
        for k in range(10):
            for l in range(10):
                for m in range(10):
                    MyArray[i,j,k,l,m] = l

As you can see, the elements shall only depend on one of the dimension indices. I tried using numpy.tile but couldn't figure it out so far.

Upvotes: 2

Views: 196

Answers (4)

Divakar
Divakar

Reputation: 221564

Here's an approach with initialization -

n = 10
a = np.empty((n,n,n,n,n),dtype=int)
a[...] = np.arange(n)[:,None]

Here's another NumPy strides based approach -

r = np.arange(n)
s = r.strides[0]
shp = (n,n,n,n,n)
out = np.lib.index_tricks.as_strided(r, shape=shp, strides=(0,0,0,s,0))

Runtime test

Approaches -

# @Eric's soln1
def broadcast_to_based(n): # Creates a read-only array
    l_vals = np.arange(n).reshape(1, 1, 1, -1, 1)
    return np.broadcast_to(l_vals, (n, n, n, n, n))

# @Eric's soln2    
def tile_based(n):
    l_vals = np.arange(n).reshape(1, 1, 1, -1, 1)
    return np.tile(l_vals, (n, n, n, 1, n))

# @kmichael08's soln          
def fromfunc_based(n):
    return np.fromfunction(lambda i, j, k, l, m : l, (n, n, n, n, n))

# @Tw UxTLi51Nus's soln
def loop_based(n): 
    MyArray = np.zeros([n,n,n,n,n],dtype=int)
    for l in range(n):
        MyArray[:, :, :, l, :] = l
    return MyArray

# Proposed-1 in this post      
def initialization_based(n):
    a = np.empty((n,n,n,n,n),dtype=int)
    a[...] = np.arange(n)[:,None]
    return a

# Proposed-2 in this post      
def strided_based(n):
    r = np.arange(n)
    s = r.strides[0]
    shp = (n,n,n,n,n)
    return np.lib.index_tricks.as_strided(r, shape=shp, strides=(0,0,0,s,0))

Timings -

In [153]: n = 10
     ...: %timeit broadcast_to_based(n)
     ...: %timeit tile_based(n)
     ...: %timeit fromfunc_based(n)
     ...: %timeit loop_based(n)
     ...: %timeit initialization_based(n)
     ...: %timeit strided_based(n)
     ...: 
100000 loops, best of 3: 4.1 µs per loop
1000 loops, best of 3: 236 µs per loop
1000 loops, best of 3: 645 µs per loop
10000 loops, best of 3: 180 µs per loop
10000 loops, best of 3: 89.1 µs per loop
100000 loops, best of 3: 5.44 µs per loop

In [154]: n = 20
     ...: %timeit broadcast_to_based(n)
     ...: %timeit tile_based(n)
     ...: %timeit fromfunc_based(n)
     ...: %timeit loop_based(n)
     ...: %timeit initialization_based(n)
     ...: %timeit strided_based(n)
     ...: 
100000 loops, best of 3: 4.05 µs per loop
100 loops, best of 3: 8.16 ms per loop
10 loops, best of 3: 24.1 ms per loop
100 loops, best of 3: 6.07 ms per loop
100 loops, best of 3: 2.31 ms per loop
100000 loops, best of 3: 5.48 µs per loop

Upvotes: 3

akoeltringer
akoeltringer

Reputation: 1721

You can do this with one loop:

import numpy as np
MyArray = np.zeros([10,10,10,10,10])

for l in range(10):
    MyArray[:, :, :, l, :] = l

Obviously you can do this also as a list comprehension.

Upvotes: 3

Eric
Eric

Reputation: 97581

Looks like you're after np.broadcast_to:

# build an array where l_vals[0,0,0,i,0] = i
l_vals = np.arange(10).reshape(1, 1, 1, -1, 1)

# duplicate that array, without copies, in the other axes
l_grid = np.broadcast_to(l_vals, (10, 10, 10, 10, 10))

It's worth noting that broadcast_to returns a readonly array, because elements actually share memory locations. If you want to write into this after creating it, then you can either call np.copy, or use tile instead:

l_grid = np.tile(l_vals, (10, 10, 10, 1, 10))

You could also just have flattened your loops:

MyArray = np.zeros([10,10,10,10,10])
for l in range(10):
    MyArray[:,:,:,l,:] = l

Upvotes: 4

kmichael08
kmichael08

Reputation: 81

Try np.fromfunction, in your case it's going to be something like that: MyArray = np.fromfunction(lambda i, j, k, l, m : l, (10, 10, 10, 10, 10))

Upvotes: 2

Related Questions