CODEWITHSUNDEEP
pythonnumpy
Jesh Kundem
Jesh Kundem

Reputation: 974

Faster way to modify a numpy array

I have a large numpy array with 4 million rows and 4 columns (shape = (4000000,4))

I need to modify/ decrease the number of rows, based on the value in fourth column. For example few of my rows in my data set look like the following: 

a = np.array([[1.32, 24.42, 224.21312, 0],[1.32, 24.42, 224.21312, 0],[1.32, 24.42, 224.21312, 1],[1.32, 24.42, 224.21312, 1],[1.32, 24.42, 224.21312, 0]]);

My result should be the following (only rows with last column value = 1)

b = [1.32, 24.42, 224.21312, 1],[1.32, 24.42, 224.21312, 1]

A for loop to go through each row is taking a long time to process.

I have 200 of these arrays, so I am already using multiprocessing for each array.

Looking for suggestions.

Upvotes: 1

Views: 515

Answers (2)

Rarblack
Rarblack

Reputation: 4664

You can convert it to dataframe and operate your operations there and then convert back to array:

df = pd.DataFrame(a)
df = df[df[3] == 1]
a = df.as_matrix()

Output:

array([[  1.32   ,  24.42   , 224.21312,   1.     ],
       [  1.32   ,  24.42   , 224.21312,   1.     ]])

Upvotes: 0

Vince W.
Vince W.

Reputation: 3785

does this work for you?

a[a[:,3] == 1]

gives:

array([[  1.32   ,  24.42   , 224.21312,   1.     ],
       [  1.32   ,  24.42   , 224.21312,   1.     ]])

Upvotes: 3

Related Questions