CODEWITHSUNDEEP
pythonpandas
abcd
abcd

Reputation: 10791

Changing certain values in multiple columns of a pandas DataFrame at once

Suppose I have the following DataFrame:

In [1]: df
Out[1]:
  apple banana cherry
0     0      3   good
1     1      4    bad
2     2      5   good

This works as expected:

In [2]: df['apple'][df.cherry == 'bad'] = np.nan
In [3]: df
Out[3]:
  apple banana cherry
0     0      3   good
1   NaN      4    bad
2     2      5   good

But this doesn't:

In [2]: df[['apple', 'banana']][df.cherry == 'bad'] = np.nan
In [3]: df
Out[3]:
  apple banana cherry
0     0      3   good
1     1      4    bad
2     2      5   good

Why? How can I achieve the conversion of both the 'apple' and 'banana' values without having to write out two lines, as in

In [2]: df['apple'][df.cherry == 'bad'] = np.nan
In [3]: df['banana'][df.cherry == 'bad'] = np.nan

Upvotes: 26

Views: 38724

Answers (3)

NL23codes
NL23codes

Reputation: 1219

While this question is broad, the answers seem very specific and not very versatile. This is just to clarify...

df = pandas.DataFrame({'Test1' :[1,2,3,4,5], 'Test2': [3,4,5,6,7], 'Test3': [5,6,7,8,9]})

   Test1 Test2 Test3
0  1     3     5
1  2     4     6
2  3     5     7
3  4     6     8
4  5     7     9

# When the index or row you want to edit is known
df.loc[3, ['Test1', 'Test2', 'Test3'] = [10, 12, 14]

# When you don't know the index but can find it by looking in a column for a specific value

df.loc[df[df['Test1'] == 4].index[0], ['Test1', 'Test2', 'Test3']] = [10, 12, 14]

   Test1 Test2 Test3
0  1     3     5
1  2     4     6
2  3     5     7
3  10    12    14
4  5     7     9

Both methods allow you to change the values of multiple columns in one line of code.

Upvotes: 1

roman
roman

Reputation: 117636

It's because df[['apple', 'banana']][df.cherry == 'bad'] = np.nan assigning to the copy of DataFrame. Try this:

df.ix[df.cherry == 'bad', ['apple', 'banana']] = np.nan

Upvotes: 5

Andy Hayden
Andy Hayden

Reputation: 375925

You should use loc and do this without chaining:

In [11]: df.loc[df.cherry == 'bad', ['apple', 'banana']] = np.nan

In [12]: df
Out[12]: 
   apple  banana cherry
0      0       3   good
1    NaN     NaN    bad
2      2       5   good

See the docs on returning a view vs a copy, if you chain the assignment is made to the copy (and thrown away) but if you do it in one loc then pandas cleverly realises you want to assign to the original.

Upvotes: 39

Related Questions