CODEWITHSUNDEEP
pythonpandas
Krishna Moorthy
Krishna Moorthy

Reputation: 73

'float' object is not iterable using shift function

I am trying to shift from one column to another

rows = df[df.iloc[:,4].notnull()]
cols = df.iloc[:,4]
df.loc[rows,columns] = df.loc[rows,cols].shift(1, axis=1)

I have attached the dataset as image, here I also have one column with value as 100%.

It cannot shift my column, it is throwing the 'float object not iterable'

enter image description here

Upvotes: 0

Views: 63

Answers (1)

ouroboros1
ouroboros1

Reputation: 14369

I am unable to trigger the exact same error, but let me just give a dummy example to show what is going wrong, and how you may perform your operation correctly. Here's a dummy df:

import pandas as pd
import numpy as np

data = {'a': [np.nan,2,3], 'b':[1,np.nan,np.nan], 'c': [np.nan,2,3], 'd':[1,np.nan,np.nan]}
df = pd.DataFrame(data)

print(df)

     a    b    c    d
0  NaN  1.0  NaN  1.0
1  2.0  NaN  2.0  NaN
2  3.0  NaN  3.0  NaN

Goal: replace NaNs in df.d with non-NaNs from df.c. If I understand correctly, you were trying as follows:

rows = df[df.iloc[:,2].notnull()]
cols = df.iloc[:,2]
df.loc[rows,cols] = df.loc[rows,cols].shift(1, axis=1)

In this particular case, the last line will throw this error:

KeyError: "None of [Index([('a',), ('b',), ('c',), ('d',)], dtype='object')] are in the [index]"

(side note: your own error is perhaps related to df.loc[rows,columns], what is columns? You mean cols surely.)

Now, in this example rows is actually a pd.DataFrame, and cols is a pd.Series, so it already goes wrong on the selection. You might think, you could fix this by using df.loc[rows.index,cols.name], but this simply generates a new pd.Series:

print(df.loc[rows.index,cols.name])

1    2.0
2    3.0
Name: c, dtype: float64

For obivous reasons, you cannot shift a pd.Series columnwise, because it only has axis 0, i.e. it is one-dimensional.

So, let's try a slightly different approach:

col = 2 # column 'c'
rows = df[df.iloc[:,col].notnull()].index # Int64Index([1, 2], dtype='int64')

# now select from the df col c AND following (colon) and assign it the columnwise shift
df.iloc[rows,col:] = df.iloc[rows,col:].shift(1, axis=1)

print(df)

     a    b   c    d
0  NaN  1.0 NaN  1.0
1  2.0  NaN NaN  2.0
2  3.0  NaN NaN  3.0

Imagine that we only want to shift non-NaNs from df.a to replace NaNs in df.b, then we write:

col = 0 # column 'a'
rows = df[df.iloc[:,col].notnull()].index # Int64Index([1, 2], dtype='int64')

# now select from the df col a AND b (+2)
df.iloc[rows,col:col+2] = df.iloc[rows,col:col+2].shift(1, axis=1)

print(df)

    a    b    c    d
0 NaN  1.0  NaN  1.0
1 NaN  2.0  2.0  NaN
2 NaN  3.0  3.0  NaN

Hope that makes some sense, and offers a solution.

Upvotes: 1

Related Questions