Split a pandas dataframe into two dataframes efficiently based on some condition

Question

So, I want to split a given dataframe into two dataframes based on an if condition for a particular column. I am currently achieving this by iterating over the whole dataframe two times. Please suggest some ways to improve this.

 player   score
 dan       10
 dmitri    45
 darren    15
 xae12     40

Like in the above dataframe, I want to split the df into two such that one df contains rows with the players scoring less than 15 and the other df contains the remaining rows. I want to this with one iteration only. (Also, if the answer can be generic for n dfs, it will help me a lot) Thanks in advance.

Answer 1

There are two possible ways to select dataframe as per your requirement. I am doing time analysis in the same query, so that we can get the idea which one is faster.

1) Using two times df

%%time

dataframe1  = dataframe[dataframe['score']>15]
dataframe2  = dataframe[dataframe['score']<=15]

the output is coming in Wall time: 4.06 ms

2) Using boolean and tilde concept:

%%time

a = dataframe.score>15

dataframe1 = dataframe[a]
dataframe2 = dataframe[~a]

This query is giving output in Wall time: 0.02 ms

Clearly the second method is much faster.

Answer 2

IICU

Use boolean select

m=df.score>15

Lessthan15=df[~m]
Morethan15=df[m]

Morethan15

LessThan15

Answer 3

Try this:

df_less_than_15 = df[df['score'] < 15]
df_more_than_15 = df[df['score'] >= 15]

you can use the same thing for every given data frame.