Reputation: 123
I have a dataframe containing sensor data. The sensor data has fluctuations in it. I would like to minimise these fluctuations to make it fit for further analysis.
The sample of the data I have looks as below:
0
------
0 5
1 5
2 0
3 5
4 5
5 0
6 0
7 0
8 0
9 0
10 1
11 1
12 0
13 1
14 1
15 0
16 3
17 3
18 3
19 0
20 0
21 0
22 0
The zero values in between non-zero values are result of fluctuation. And I would like to have these zero values which are in between non-zero values to be replaces with the neighbouring non-zero values. This should not affect the trailing and leading zero values which are present in the column of the sensor data.
My desired output should look like:
0
------
0 5
1 5
2 5 <---
3 5
4 5
5 0
6 0
7 0
8 0
9 0
10 1
11 1
12 1 <---
13 1
14 1
15 1 <---
16 3
17 3
18 3
19 0
20 0
21 0
22 0
Arrows indicate my required replacments. What would be the best way to do it?
Upvotes: 1
Views: 1324
Reputation: 65
1st Method
Steps:
2nd Method
Use ffill() method in dataframe and copy previous value if the present value is 0.
Upvotes: 1
Reputation: 862581
Replace 0
to missing values and then forward filling missing values with limit
parameter, but only for rows if not missing values for forward and back filling values:
s = df[0].replace(0, np.nan)
ff = s.ffill(limit=1)
bf = s.bfill(limit=1)
df['new'] = np.where(ff.notna() & bf.notna(), ff, 0).astype(int)
print (df)
0 new
0 5 5
1 5 5
2 0 5
3 5 5
4 5 5
5 0 0
6 0 0
7 0 0
8 0 0
9 0 0
10 1 1
11 1 1
12 0 1
13 1 1
14 1 1
15 0 1
16 3 3
17 3 3
18 3 3
19 0 0
20 0 0
21 0 0
22 0 0
Upvotes: 1