Sash7
Sash7

Reputation: 123

How to replace zero values in between some non-zero values in pandas?

I have a dataframe containing sensor data. The sensor data has fluctuations in it. I would like to minimise these fluctuations to make it fit for further analysis.

The sample of the data I have looks as below:

    0
------
0   5
1   5
2   0
3   5
4   5
5   0
6   0
7   0
8   0
9   0
10  1
11  1
12  0
13  1
14  1
15  0
16  3
17  3
18  3
19  0
20  0
21  0
22  0

The zero values in between non-zero values are result of fluctuation. And I would like to have these zero values which are in between non-zero values to be replaces with the neighbouring non-zero values. This should not affect the trailing and leading zero values which are present in the column of the sensor data.

My desired output should look like:

    0
------
0   5
1   5
2   5 <---
3   5
4   5
5   0
6   0
7   0
8   0
9   0
10  1
11  1
12  1 <---
13  1
14  1
15  1 <---
16  3
17  3
18  3
19  0
20  0
21  0
22  0

Arrows indicate my required replacments. What would be the best way to do it?

Upvotes: 1

Views: 1324

Answers (2)

Saksham
Saksham

Reputation: 65

1st Method

Steps:

  1. Convert the column into list.
  2. Iterate through list , check in each index if the value is 0, if yes make the present index value same as previous else iterate through next.
  3. Make the new list as a column in dataframe.

2nd Method

Use ffill() method in dataframe and copy previous value if the present value is 0.

Upvotes: 1

jezrael
jezrael

Reputation: 862581

Replace 0 to missing values and then forward filling missing values with limit parameter, but only for rows if not missing values for forward and back filling values:

s = df[0].replace(0, np.nan)
ff = s.ffill(limit=1)
bf = s.bfill(limit=1)

df['new'] = np.where(ff.notna() & bf.notna(), ff, 0).astype(int)
print (df)
    0  new
0   5    5
1   5    5
2   0    5
3   5    5
4   5    5
5   0    0
6   0    0
7   0    0
8   0    0
9   0    0
10  1    1
11  1    1
12  0    1
13  1    1
14  1    1
15  0    1
16  3    3
17  3    3
18  3    3
19  0    0
20  0    0
21  0    0
22  0    0

Upvotes: 1

Related Questions