CODEWITHSUNDEEP
python-3.xpandas
Danish
Danish

Reputation: 2871

Fill NaN values based on specific condition in pandas

I have a dataframe as shown below

Date                t_factor     t1         t2         t3       t_function
2020-02-01             5          4         NaN        NaN      4
2020-02-03             23         6         NaN        NaN      6
2020-02-06             14         9         NaN        NaN      9
2020-02-09             23         NaN       NaN        NaN      0
2020-02-10             23         NaN       NaN        NaN      0
2020-02-11             23         NaN       NaN        NaN      0
2020-02-13             30         NaN       3          NaN      3   
2020-02-20             29         NaN       66         NaN      66
2020-02-29             100        NaN       291        NaN      291
2020-03-01             38         NaN       NaN        NaN      0
2020-03-10             38         NaN       NaN        NaN      0
2020-03-11             38         NaN       NaN        4        4 
2020-03-26             70         NaN       NaN        4        4
2020-03-29             70         NaN       NaN        4        4

In which I would like to fill NaN values after non NaN value as last NaN value of that column

Here the columns I wanted to impute are t1, t2 and t3.

Expected Output

Date                t_factor     t1       t2         t3       t_function
2020-02-01             5          4       NaN        NaN      4
2020-02-03             23         6       NaN        NaN      6
2020-02-06             14         9       NaN        NaN      9
2020-02-09             23         9       NaN        NaN      0
2020-02-10             23         9       NaN        NaN      0
2020-02-11             23         9       NaN        NaN      0
2020-02-13             30         9       3          NaN      3   
2020-02-20             29         9       66         NaN      66
2020-02-29             100        9       291        NaN      291
2020-03-01             38         9       291        NaN      0
2020-03-10             38         9       291        NaN      0
2020-03-11             38         9       291        4        4 
2020-03-26             70         9       291        4        4
2020-03-29             70         9       291        4        4

Upvotes: 0

Views: 213

Answers (3)

Shubham Sharma
Shubham Sharma

Reputation: 71689

Use ffill:

df[['t1', 't2', 't3']] = df[['t1', 't2', 't3']].ffill()

Result:

          Date  t_factor   t1     t2   t3  t_function
0   2020-02-01         5  4.0    NaN  NaN           4
1   2020-02-03        23  6.0    NaN  NaN           6
2   2020-02-06        14  9.0    NaN  NaN           9
3   2020-02-09        23  9.0    NaN  NaN           0
4   2020-02-10        23  9.0    NaN  NaN           0
5   2020-02-11        23  9.0    NaN  NaN           0
6   2020-02-13        30  9.0    3.0  NaN           3
7   2020-02-20        29  9.0   66.0  NaN          66
8   2020-02-29       100  9.0  291.0  NaN         291
9   2020-03-01        38  9.0  291.0  NaN           0
10  2020-03-10        38  9.0  291.0  NaN           0
11  2020-03-11        38  9.0  291.0  4.0           4
12  2020-03-26        70  9.0  291.0  4.0           4
13  2020-03-29        70  9.0  291.0  4.0           4

Upvotes: 1

quest
quest

Reputation: 3926

Here is how I will go:

def fill_na(col):
    ind = df[col].last_valid_index()    
    df[col][ind+1:].fillna(df[col][ind], inplace=True)

fill_na('t1')
fill_na('t2')
fill_na('t3')

Upvotes: 1

Divyessh
Divyessh

Reputation: 2721

We can define a function for that

def imporove(iterable):
    for i in range(len(iterable)):
        if iterable[i].isnull() == True:
            iterable[i] = iterable[i-1]

I hope you got a basic idea. now you can pass df['t1'].apply(improve)

Upvotes: 1

Related Questions