CODEWITHSUNDEEP
pythonarraysnumpy
Thomas Leonard
Thomas Leonard

Reputation: 1047

complementary slicing in a numpy array

If I have a numpy array for example :

A = np.array([[3, 2], [2, -1], [2, 3], [5, 6], [7,-1] , [8, 9]])

I would like to separate the part of the array with the subarrays having -1 from the ones who don't. Keep in mind that I'm working on very big data set, so every operation can be very long so I try to have the most effective way memory and CPU-time wise.

What I am doing for the moment is :

 slicing1 = np.where(A[:, 1] == -1)
 with_ones = A[slicing1]
 slicing2 = np.setdiff1d(np.arange(A.shape[0]), slicing1, assume_unique=True)
 without_ones = A[slicing2]

Is there a way to not create the slicing2 list to decrease the memory consumption as it can be very big? Is there a better way to approach the problem?

Upvotes: 3

Views: 3435

Answers (3)

Balint Domokos
Balint Domokos

Reputation: 1021

Or you could use a generator function:

A = np.array([[3, 2], [2, -1], [2, 3], [5, 6], [7,-1] , [8, 9]])

def filt(arr):
    for item in arr:
        if item[1]!=-1:
            yield item

new_len = 0
for item in A:
    if item[1] != -1:
        new_len += 1

without_ones = np.empty([new_len, 2], dtype=int)
for i, item in enumerate(filt(A)): 
    without_ones[i] = item

Upvotes: 1

ely
ely

Reputation: 77404

One way is to store the logical index needed and then in the second case index using its logical negation:

In [46]: indx = A[:, 1] != -1

In [47]: A[indx]
Out[47]: 
array([[3, 2],
       [2, 3],
       [5, 6],
       [8, 9]])

In [48]: A[~indx]
Out[48]: 
array([[ 2, -1],
       [ 7, -1]])

Upvotes: 6

S4M
S4M

Reputation: 4661

I managed to create without_ones with:

filter(lambda x: x[1] != -1,A)

Upvotes: 1

Related Questions