CODEWITHSUNDEEP
pythonpandas
ben890
ben890

Reputation: 1133

Remove element from list in pandas dataframe based on value in column

Let's say I have following dataframe:

a = [[1,2,3,4,5,6],[23,23,212,223,1,12]]
b = [1,1]


df = pd.DataFrame(zip(a,b), columns = ['a', 'b'])

And my goal is to remove the elements in the lists in series A that are in series B. My attempt at doing so is below:

df['a'] = [i.remove(j) for i,j in zip(df.a, df.b)]

The logic seems sounds to me however I'm ending up with df['a'] being a series of nulls. What is going on here?

Upvotes: 11

Views: 21045

Answers (5)

LocoGris
LocoGris

Reputation: 4480

How about this:

b = [[1],[1]] 

df['a'] = df.apply(lambda row: list(set(row['a']).difference(set(row['b']))), axis=1)

b must be in this way, but you can get the difference even if you want to remove more than an element.

Example:

import pandas as pd
a = [[1,2,3,4,5,6],[23,23,212,223,1,12]]
b = [[1,5],[1,23]]


df = pd.DataFrame(zip(a,b), columns = ['a', 'b'])



df['a'] = df.apply(lambda row: list(set(row['a']).difference(set(row['b']))), axis=1)

Upvotes: 0

AChampion
AChampion

Reputation: 30288

Here's an alternative way of doing it:

In []:
df2 = df.explode('a')
df['a'] = df2.a[df2.a != df2.b].groupby(level=0).apply(list)
df

Out[]:
                        a  b
0         [2, 3, 4, 5, 6]  1
1  [23, 23, 212, 223, 12]  1

Upvotes: 6

BENY
BENY

Reputation: 323366

What I will do 

s=pd.DataFrame(df.a.tolist(),index=df.index)
df['a']=s.mask(s.eq(df.b,0)).stack().astype(int).groupby(level=0).apply(list)
Out[264]: 
0           [2, 3, 4, 5, 6]
1    [23, 23, 212, 223, 12]
dtype: object

Upvotes: 2

Ajit Wadalkar
Ajit Wadalkar

Reputation: 170

list.remove(x) removes the value in-place and returns None. That is why the above code is failing for you. You can also do something like the following.

a = [[1,2,3,4,5,6],[23,23,212,223,1,12]]
b = [1,1]
df = pd.DataFrame(zip(a,b), columns = ['a', 'b'])
for i, j in zip(df.a, df.b):
    i.remove(j)

print df

                        a  b
0         [2, 3, 4, 5, 6]  1
1  [23, 23, 212, 223, 12]  1

Upvotes: 5

Celius Stingher
Celius Stingher

Reputation: 18377

Assuming row b only contains one value, then you can try with the following using a list comprehension within a function, and then simply apply it:

import pandas as pd
a = [[1,2,3,4,5,6],[23,23,212,223,1,12]]
b = [1,1]


df = pd.DataFrame(zip(a,b), columns = ['a', 'b'])
def removing(row):
    val = [x for x in row['a'] if x != row['b']]
    return val
df['c'] = df.apply(removing,axis=1)
print(df)

Output:

                           a  b                       c
0         [1, 2, 3, 4, 5, 6]  1         [2, 3, 4, 5, 6]
1  [23, 23, 212, 223, 1, 12]  1  [23, 23, 212, 223, 12]

Upvotes: 2

Related Questions