CODEWITHSUNDEEP
pythonpandasdataframe
Kelaref
Kelaref

Reputation: 517

Pandas: Delete Rows From DataFrame Matching Conditions

I have a Pandas DataFrame as follows:

ID   PROD   QTY   PRICE   FEES
1     G      2     120    -1.2   
2     B      5     150    -1.5
3     S      2     80     -2.0   
4     T      5     300    +1.0
1     G     -2     120    +1.2   
2     B     -5     150    +1.5

I am hoping to delete rows where ALL the following conditions are met:

1) They have equal ID

2) They have equal price

3) They have opposite QTY

4) They have opposite Fees

The desired result is the following:

ID   PROD   QTY   PRICE   FEES
3     S      2     80     -2.0   
4     T      5     300    +1.0

My first instinct is to sort the dataframe by ID & price, and then iterate the dataframe, but I am searching for a more pythonic more efficient approach.

Perhaps a solution might require a group by ID & price, then delete where fees and qty are equal to zero.

Thank you

Upvotes: 0

Views: 1291

Answers (2)

Allen Qin
Allen Qin

Reputation: 19957

Setup

df=pd.DataFrame({'FEES': {0: -1.2, 1: -1.5, 2: -2.0, 3: 1.0, 4: 1.2, 5: 1.5},
 'ID': {0: 1, 1: 2, 2: 3, 3: 4, 4: 1, 5: 2},
 'PRICE': {0: 120, 1: 150, 2: 80, 3: 300, 4: 120, 5: 150},
 'PROD': {0: 'G', 1: 'B', 2: 'S', 3: 'T', 4: 'G', 5: 'B'},
 'QTY': {0: 2, 1: 5, 2: 2, 3: 5, 4: -2, 5: -5}})

Solution

#define a list to store duplicates index
dups=[]

#apply conditions to locate rows to be removed.
df.apply(lambda x: dups.extend(df.loc[(df.ID==x.ID)&(df.PRICE==x.PRICE)&(df.QTY+x.QTY==0)&(df.FEES+x.FEES==0)].index.tolist()), axis=1)

#filter results based on dups ID
df.loc[~df.index.isin(dups)]
Out[122]: 
   ID PROD  QTY  PRICE  FEES
2   3    S    2     80  -2.0
3   4    T    5    300   1.0

Upvotes: 1

A.Kot
A.Kot

Reputation: 7913

To get part one, you can first remove all duplicates based upon ID and Price:

df.drop_duplicates(subset = ['ID', 'PRICE'], inplace=True)

Then you want to groupby all IDs to identify total Quantity and total Fees:

 df = df.groupby('ID', as_index=False).sum()

You can then filter out anything with sum 0

df[df.QTY != 0]

Upvotes: 3

Related Questions