CODEWITHSUNDEEP
pythonpandasdataframe
Hrithik Diwakar
Hrithik Diwakar

Reputation: 129

pandas.dataframe.fillna: NotImplementedError

I wanted to impute a dataframe using the fillna command in pandas. Here is a snippet of my code:

import glob
import pandas as pd
files=glob.glob("IN.201*.csv")
i=0
n=1    
#the while loops are for reading and writing different subsets of the table into 
#different .txt files:
while i<15:
   j=0
   while j<7:
    dfs=[]
    m=1
   #for loop over only one file for testing:
       for file in files[:1]:
        z=i+1
        #reading subset of the dataframe:
        k=float(68.109375)+float(1.953125)*i
        k1=float(68.109375)+float(1.953125)*z
        l=float(8.0)+float(4)*j
        l1=float(8.0)+float(4)*(j+1)
        df=pd.read_csv(path+file).query( '@k <= lon < @k1 and @l < lat <= @l1')[['lon','lat','country','avg']]
        #renaming columns in df: 
        df.rename(columns={"avg":"Day"+str(m)}, inplace=True)
        #print(df)
        m=m+1
        dfs.append(df)
  #imputation: 
    df_final=dfs[0].fillna(method='bfill', axis='columns', inplace=True).fillna(method='ffill', axis=1, inplace=True)
  #writing to a txt file:
    with open('Region_'+str(n), 'w+') as f:
        df_final.to_csv(f)

    n=n+1
    j=j+1
   i=i+1

Error:

 Traceback (most recent call last):
 File "imputation_test.py", line 42, in <module>
 df_final=dfs[0].fillna(method='bfill', axis='columns', inplace=True).fillna(
 method='ffill', axis=1, inplace=True)
 File "C:\Users\DELL\AppData\Local\Programs\Python\Python36\lib\site- 
 packages\p
 andas\core\frame.py", line 3787, in fillna
 downcast=downcast, **kwargs)
 File "C:\Users\DELL\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\core\generic.py", line 5359, in fillna
raise NotImplementedError()
 NotImplementedError

Motivation for the code:

I essentially wanted to read a .csv file into multiple dataframes consisting of different subsets of this table, (hence all the loops that I have used) in order to rearrange and split the .csv file/s(actually I want to do this for multiple .csv files) into a more suitable format. I wanted to then fill the missing data using the fillna command along the column axis.

The code is structured for reading into multiple .csv files and thus has unnecessary commands like 'df=[ ]' and the 'for loop', but for simplification purposes I was trying out this code first just to make sure and I got this error. Feel free to ask for more info for this error. Thanks!

Upvotes: 2

Views: 2891

Answers (2)

John Velonis
John Velonis

Reputation: 1649

I came across this error when using ffill (which is a synonym for fillna(method='ffill')) to fill missing values across columns with inplace=True when the data frame contains both ints and NaNs in the same row.

The workaround is to use inplace=False:

df = pd.DataFrame(data={'a': [1]})
df = df.reindex(columns=['a', 'b', 'c'])  # columns b and c contain NaN
df.ffill(axis='columns', inplace=True)  # raises NotImplementedError
df = df.ffill(axis='columns')  # works (inplace defaults to False)

Upvotes: 0

Scott Boston
Scott Boston

Reputation: 153510

Use bfill and ffill with axis=1:

dfs = dfs.bfill(axis=1).ffill(axis=1)

Part of the problem are the inplace=True and chaining methods. inplace=True returns a null object so, there is nothing to call chained methods from. The second part is that fillna(method='ffill') can be shortened to just ffill().

Upvotes: 5

Related Questions