Effectively group array by element

Question

Lets say i have

lags = [0, 30, 60, 90, 120, 150, 180, np.inf]

and

list = [[500, 800, 1000, 200, 1500], [220, 450, 350, 1070, 1780], [900, 450, 1780, 1450, 100], 
        [340, 670, 830, 1370, 1420], [850, 630, 1230, 1670, 910]]

angle = [[50, 80, 100, 20, 150], [22, 45, 35, 107, 178], [90, 45, 178, 145, 10], 
        [34, 67, 83, 137, 142], [85, 63, 123, 167, 91]]

I would like to get each element in list and store it in a different seperate array depending on its value such that;

for all list.values where angles.value is less than 30
list1 = [200, 220, 100]
for all list.values where angles.value is between 30 and 60
list2 = [500, 450, 350, 450, 340] 
for all list.values where angles.value is between 60 and 90
list3 = [800, 670, 830, 850, 630]

etc..

I did something like this:

sortlist = defaultdict(list)
ulist = np.unique(list)
uangle = np.unique(angle)
for lag in lags:
    count += 1
    for k, dummy_val in enumerate(uangle):
        if lag <= uangle[k] < lag + 1:
            sortlist[count].append(ulist[k])

I am wondering if there is a pythonic/effective way in doing this to increase performance.

Answer 1

With numpy:

import numpy as np

lags = [0, 30, 60, 90, 120, 150, 180, np.inf]
alist = np.array([[500, 800, 1000, 200, 1500], [220, 450, 350, 1070, 1780], [900, 450, 1780, 1450, 100], 
        [340, 670, 830, 1370, 1420], [850, 630, 1230, 1670, 910]])
angle = np.array([[50, 80, 100, 20, 150], [22, 45, 35, 107, 178], [90, 45, 178, 145, 10], 
        [34, 67, 83, 137, 142], [85, 63, 123, 167, 91]])

i=0
while i<len(lags)-1:
    print alist[(lags[i] <= angle) & (angle < lags[i+1] )]
    i+=1
    

Output:

[200, 220, 100]
[500, 450, 350, 450, 340]
[800, 670, 830, 850, 630]
[1000, 1070, 900, 910]
[1450, 1370, 1420, 1230]
[1500, 1780, 1780, 1670]
[]
    

angle<lags[i] will create a boolean index which will mask the undesired values in alist.

---

With zip() and list comprehension:

import numpy as np

lags = [0, 30, 60, 90, 120, 150, 180, np.inf]

alist = [[500, 800, 1000, 200, 1500], [220, 450, 350, 1070, 1780], [900, 450, 1780, 1450, 100], 
        [340, 670, 830, 1370, 1420], [850, 630, 1230, 1670, 910]]

angle = [[50, 80, 100, 20, 150], [22, 45, 35, 107, 178], [90, 45, 178, 145, 10], 
        [34, 67, 83, 137, 142], [85, 63, 123, 167, 91]]

i=0
while i<len(lags)-1:
    print [b[0] for a in zip(alist, angle) for b in zip(*a) if lags[i]<= b[1] < lags[i+1]]
    i+=1
    

Output:

[200, 220, 100]
[500, 450, 350, 450, 340]
[800, 670, 830, 850, 630]
[1000, 1070, 900, 910]
[1450, 1370, 1420, 1230]
[1500, 1780, 1780, 1670]
[]

Answer 2

Here's a vectorized approach -

an = angle.ravel()
sidx = an.argsort()
cut_idx = np.searchsorted(an[sidx], lags)
out = np.split(list1.ravel()[sidx], cut_idx[1:-1])

Sample input, output -

In [97]: lags = np.array([0, 30, 60, 90, 120, 150, 180, np.inf])
    ...: 
    ...: list1 = np.array([[500, 800, 1000, 200, 1500], \
    ...:                   [220, 450, 350, 1070, 1780], \
    ...:                   [900, 450, 1780, 1450, 100], 
    ...:                   [340, 670, 830, 1370, 1420], \
    ...:                   [850, 630, 1230, 1670, 910]])
    ...: 
    ...: angle = np.array([[50, 80, 100, 20, 150],\
    ...:                   [22, 45, 35, 107, 178],\
    ...:                   [90, 45, 178, 145, 10], 
    ...:                   [34, 67, 83, 137, 142],\
    ...:                   [85, 63, 123, 167, 91]])
    ...: 

In [99]: out
Out[99]: 
[array([100, 200, 220]),            # <----- 0 to 30
 array([340, 350, 450, 450, 500]),  # <----- 30 to 60
 array([630, 670, 800, 830, 850]),  # <----- 60 to 90
 array([ 900,  910, 1000, 1070]),   # <----- 90 to 120
 array([1230, 1370, 1420, 1450]),   # <----- 120 to 150
 array([1500, 1670, 1780, 1780]),   # <----- 150 to 180
 array([], dtype=int64)]            # <----- 180 to Inf

Answer 3

Solution based on achieving it in Plain Python (without numpy)

Instead of storing it in separate variables, you may store the values within a dict (usage of collections.defaultdict is a better choice).

You may create a function to return the group based on angle:

def get_group_from_angle(angle):
    group = ''
    if angle < 30:
        group = 'a'
    elif 30 < angle < 60:
        group = 'b' 
    elif 60 < angle < 90:
        group = 'c'
    return group

Then use the above function within for to create the desired dict as:

from collections import defaultdict
my_dict = defaultdict(list)

#  `alist` and `angle` are variables holding values as mentioned in Question

for ll, aa in zip(list, angle):
    for l, a in zip(ll, aa):
        my_dict[get_group_from_angle(a)].append(l)

The final value hold by my_dict will be:

{ 
    'a': [200, 220, 100], 
    'b': [500, 450, 350, 450, 340],
    'c': [800, 670, 830, 850, 630], 
    '': [1000, 1500, 1070, 1780, 900, 1780, 1450, 1370, 1420, 1230, 1670, 910]
    # ^ number whose angle is not present in any specified range 
}