CODEWITHSUNDEEP
pythonpandas
asiu321
asiu321

Reputation: 23

Fill in np.nan values with the value of the next occurrance of non np.nan value

I am trying to figure out how to replace all nan values under a certain condition. If a value is nan, I would like to have it replaced with the date to the right side of it in it's specific list. If there are no date value to the right of the nan value, I would like to leave it as is.

Here is my starting dataframe.

mydf = [['2019-01-30', nan, nan, nan, '2020-03-09'],
        ['2018-11-29', nan, '2019-06-24', '2019-12-18', '2020-02-11'],
        [nan, nan, '2020-02-25', nan, nan]]

I would like this to end up looking like this.

mydf = [['2019-01-30', '2020-03-09', '2020-03-09', '2020-03-09', '2020-03-09'],
        ['2018-11-29', '2019-06-24', '2019-06-24', '2019-12-18', '2020-02-11'],
        ['2020-02-25', '2020-02-25', '2020-02-25', nan, nan]]

Here is my current attempt:

for i in range(0,len(mydf)):
    for j, k in enumerate(mydf[i]):
        if k is np.nan:
            mydf[i][j] = mydf[i][j+1]

mydf

but I receive an error. I can't seem to figure out how to stop the loop within each list once the remaining values in the list are all nan.

IndexError                                Traceback (most recent call last)
<ipython-input-247-3f0a1ce84ea0> in <module>
      2     for j, k in enumerate(mydf[i]):
      3         if k is np.nan:
----> 4             mydf[i][j] = mydf[i][j+1]
      5 
      6 mydf

IndexError: list index out of range

mydf
    [['2020-02-25', '2020-02-25', '2020-02-25', nan, nan],
     ['2018-11-29', nan, '2019-06-24', '2019-12-18', '2020-02-11'],
     ['2019-01-30', nan, nan, nan, '2020-03-09']]

Upvotes: 2

Views: 51

Answers (1)

lostin
lostin

Reputation: 730

You can do as desired.

mydf.fillna(method='ffill') 
mydf.fillna(method='bfill')

No need to do loop here.

Upvotes: 1

Related Questions