CODEWITHSUNDEEP
pythonpandas
komodovaran_
komodovaran_

Reputation: 2012

Create new column throws Pandas error despite following .loc recommendation

Coming from R, I naively tried

dfE_fitted['E_after'] = dfE_fitted['E_before']

That gave me

A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

Fair enough, I'll try that then:

dfE_fitted.loc[:,'E_after'] = dfE_fitted['E_before']

This gives me

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py:337: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
self.obj[key] = _infer_fill_value(value)
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pandas/core/indexing.py:517: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[item] = s

What am I doing horribly wrong?

Upvotes: 6

Views: 2912

Answers (1)

piRSquared
piRSquared

Reputation: 294488

It isn't how you are doing your assignment. It's how you constructed dfE_fitted. You constructed it in such a way so as to be a "copy" of something else. The recommendation to use .loc is intended to be used in the construction of dfE_fitted. Even then, using .loc isn't a guarantee. You can still have a "is_copy" flag after using .loc

However, I can't see how you did that. Best I can do is recommend that you do

dfE_fitted = dfE_fitted.copy()

This will detangle the "copy" relationship and you can proceed to assign as you were.

dfE_fitted['E_after'] = dfE_fitted['E_before']

Working example 

df = pd.DataFrame(dict(A=[1, 2], E_before=[3, 4], E_after=[5, 6]))

# Notice how I constructed this
dfE_fitted = df[['E_before']]

Then I try

dfE_fitted['E_after'] = dfE_fitted['E_before']

I get

enter image description here

However, had I constructed dfE_fitted like this:

# Notice I used `.loc` like what was recommended
dfE_fitted = df.loc[:, ['E_before']]

I can do the following with no warning:

dfE_fitted['E_after'] = dfE_fitted['E_before']

You can tell if a dataframe is a "copy" by looking at its is_copy attribute. If it is a copy, it will return a 

<weakref at 0x1188d94f8; to 'DataFrame' at 0x1184bf898>

Which evaluates to True

bool(dfE_fitted.is_copy)

True

Otherwise, if it isn't a "copy" it is None and evaluates to False

Upvotes: 9

Related Questions