CODEWITHSUNDEEP
pythonarraysnumpy
aiven
aiven

Reputation: 4313

Numpy: find indeces of mask edges

I'm trying to find indeces of masked segments. For example:

mask = [1, 0, 0, 1, 1, 1, 0, 0]
segments = [(0, 0), (3, 5)]

Current solution looks like this (and it's very slow, because my mask contains millions of numbers):

segments = []
start = 0
for i in range(len(mask) - 1):
    e1 = mask[i]
    e2 = mask[i + 1]
    if e1 == 0 and e2 == 1:
        start = i + 1
    elif e1 == 1 and e2 == 0:
        segments.append((start, i))

Is there any way to do this efficiently with numpy?

The only thing that i've managed to google is numpy.ma.notmasked_edges, but it doesn't look like what i need.

Upvotes: 9

Views: 1877

Answers (2)

Divakar
Divakar

Reputation: 221524

Here's one approach -

def start_stop(a, trigger_val):
    # "Enclose" mask with sentients to catch shifts later on
    mask = np.r_[False,np.equal(a, trigger_val),False]

    # Get the shifting indices
    idx = np.flatnonzero(mask[1:] != mask[:-1])

    # Get the start and end indices with slicing along the shifting ones
    return zip(idx[::2], idx[1::2]-1)

Sample run -

In [216]: mask = [1, 0, 0, 1, 1, 1, 0, 0]

In [217]: start_stop(mask, trigger_val=1)
Out[217]: [(0, 0), (3, 5)]

Use it to get the edges for 0s -

In [218]: start_stop(mask, trigger_val=0)
Out[218]: [(1, 2), (6, 7)]

Timings on 100000x scaled up datasize -

In [226]: mask = [1, 0, 0, 1, 1, 1, 0, 0]

In [227]: mask = np.repeat(mask,100000)

# Original soln
In [230]: %%timeit
     ...: segments = []
     ...: start = 0
     ...: for i in range(len(mask) - 1):
     ...:     e1 = mask[i]
     ...:     e2 = mask[i + 1]
     ...:     if e1 == 0 and e2 == 1:
     ...:         start = i + 1
     ...:     elif e1 == 1 and e2 == 0:
     ...:         segments.append((start, i))
1 loop, best of 3: 401 ms per loop

# @Yakym Pirozhenko's soln
In [231]: %%timeit
     ...: slices = np.ma.clump_masked(np.ma.masked_where(mask, mask))
     ...: result = [(s.start, s.stop - 1) for s in slices]
100 loops, best of 3: 4.8 ms per loop

In [232]: %timeit start_stop(mask, trigger_val=1)
1000 loops, best of 3: 1.41 ms per loop

Upvotes: 5

hilberts_drinking_problem
hilberts_drinking_problem

Reputation: 11602

An alternative approach with np.ma.clump_masked.

mask = np.array([1, 0, 0, 1, 1, 1, 0, 0])
# get a list of "clumps" or contiguous slices.
slices = np.ma.clump_masked(np.ma.masked_where(mask, mask))
# convert each slice to a tuple of indices.
result = [(s.start, s.stop - 1) for s in slices]
# [(0, 0), (3, 5)]

Upvotes: 2

Related Questions