Harriv
Harriv

Reputation: 6137

Creating image from point list with Numpy, how to speed up?

I've following code which seems to be performance bottleneck:

for x, y, intensity in myarr:
  target_map[x, y] = target_map[x,y] + intensity

There are multiple coordinates for same coordinate with variable intensity.

Datatypes:

> print myarr.shape, myarr.dtype
(219929, 3) uint32

> print target_map.shape, target_map.dtype
(150, 200) uint32

Is there any way to optimize this loop, other than writing it in C?

This seems to be related question, how ever I couldn't get the accepted answer working for me: How to convert python list of points to numpy image array?

I get following error message:

Traceback (most recent call last):
  File "<pyshell#38>", line 1, in <module>
    image[coordinates] = 1
IndexError: too many indices for array

Upvotes: 2

Views: 73

Answers (1)

Jaime
Jaime

Reputation: 67437

If you convert your 2D coordinates into target_map into flat indices into it using np.ravel_multi_index, you can use np.unique and np.bincount to speed things up quite a bit:

def vec_intensity(my_arr, target_map) :
    flat_coords = np.ravel_multi_index((my_arr[:, 0], my_arr[:, 1]),
                                       dims=target_map.shape)
    unique_, idx = np.unique(flat_coords, return_inverse=True)
    sum_ = np.bincount(idx, weights=my_arr[:, 2])
    target_map.ravel()[unique_] += sum_
    return target_map

def intensity(my_arr, target_map) :
    for x, y, intensity in myarr:
        target_map[x, y] += intensity
    return target_map

#sample data set
rows, cols = 150, 200
items = 219929
myarr = np.empty((items, 3), dtype=np.uint32)
myarr[:, 0] = np.random.randint(rows, size=(items,))
myarr[:, 1] = np.random.randint(cols, size=(items,))
myarr[:, 2] = np.random.randint(100, size=(items,))

And now:

In [6]: %timeit target_map_1 = np.zeros((rows, cols), dtype=np.uint32); target_map_1 = vec_intensity(myarr, target_map_1)
10 loops, best of 3: 53.1 ms per loop

In [7]: %timeit target_map_2 = np.zeros((rows, cols), dtype=np.uint32); target_map_2 = intensity(myarr, target_map_2)
1 loops, best of 3: 934 ms per loop

In [8]: np.all(target_map_1 == target_map_2)
Out[8]: True

That's almost a 20x speed increase.

Upvotes: 1

Related Questions