CODEWITHSUNDEEP
pythonarraysnumpyoptimizationindexing
dgmp88
dgmp88

Reputation: 557

Fast advanced indexing in numpy

I'm trying to take a slice from a large numpy array as quickly as possible using fancy indexing. I would be happy returning a view, but advanced indexing returns a copy.

I've tried solutions from here and here with no joy so far.

Toy data:

data = np.random.randn(int(1e6), 50)
keep = np.random.rand(len(data))>0.5

Using the default method:

%timeit data[keep] 
10 loops, best of 3: 86.5 ms per loop

Numpy take:

%timeit data.take(np.where(keep)[0], axis=0)
%timeit np.take(data, np.where(keep)[0], axis=0)
10 loops, best of 3: 83.1 ms per loop
10 loops, best of 3: 80.4 ms per loop

Method from here:

rows = np.where(keep)[0]
cols = np.arange(a.shape[1])
%timeit (a.ravel()[(cols + (rows * a.shape[1]).reshape((-1,1))).ravel()]).reshape(rows.size, cols.size)
10 loops, best of 3: 159 ms per loop

Whereas if you're taking a view of the same size:

%timeit data[1:-1:2, :]
1000000 loops, best of 3: 243 ns per loop

Upvotes: 2

Views: 1142

Answers (1)

user2357112
user2357112

Reputation: 280778

There's no way to do this with a view. A view needs consistent strides, while your data is randomly scattered throughout the original array.

Upvotes: 6

Related Questions