CODEWITHSUNDEEP
pythonpandasnumpy
Jesper - jtk.eth
Jesper - jtk.eth

Reputation: 7484

Create generic data filter on pandas dataframe in Python

I am trying to create a generic filter on some data. Consider a boiled down example:

my_data = pd.DataFrame(some_data)
my_filter = {"value": 100, "direction": ">"}

now I want to leverage numpy's where to do something like:

np.where(my_data (...) my_filter["value"])

where, as "(...)", I want my_filter["direction"]. Sure, I can use something like "eval(...)" by putting together an appropriate string.

Question But I was wondering if there is a way to do this without eval(...)?

Follow-up on "already answered" claim: The key here is that I need the direction to be generic. Sub-selecting data based on values is easy "np.where(my_data == my_filter["value"])", e.g.

Upvotes: 1

Views: 652

Answers (4)

MaxU - stand with Ukraine
MaxU - stand with Ukraine

Reputation: 210942

Demo:

In [73]: df
Out[73]:
   value direction
0    100         >
1     12         ^
2    101         >
3     13         v
4    100         <

In [74]: my_filter = ["value >= 100", "direction == '>'"]

In [75]: df.query(' and '.join(my_filter))
Out[75]:
   value direction
0    100         >
2    101         >

Upvotes: 1

CodeCollector
CodeCollector

Reputation: 476

You can use the operator module. Here is one example

import pandas as pd
import operator
my_data = pd.DataFrame([[1,1],[1,1]])
my_filter = {"value": 1, "direction": operator.ge}
print(my_data.loc[my_filter["direction"](my_data[0],my_filter["value"])])

This should ideally do the job. Otherwise you can use np.where with some modifications(personally I haven't used np.where much).

Upvotes: 1

Anton vBR
Anton vBR

Reputation: 18916

Here is an example applied with your data from Select data frame by dict passing in parameters or comprehension list python pandas

import pandas as pd

df = pd.DataFrame(dict(value=[100,200],direction=['>','<']))
my_filter = {"value":100, "direction": ">"}

# Apply
mask = (df[list(my_filter)] == pd.Series(my_filter)).all(axis=1)
df[mask]

Upvotes: 1

Lucas Dresl
Lucas Dresl

Reputation: 1170

If I get you, using np.where i imagine something like this

df['filter'] = np.where((df['value'] == 100) & (df['direction'] == '>'), 1,0)

df.loc[df['filter'] == 1]

Upvotes: 1

Related Questions