How to denoise data using wavelet transform

Question

I want to denoise the signal with wavelet transform, but somehow the data after denoising doesn't change significantly

the code:

df = pd.read_csv('0311LalaStand5Min1.csv', low_memory=False)
columns = ['Fx','Fy','Fz','Mx','My','Mz']
selected_df = df[columns]
FPDatas = selected_df[:15000]

FPData = np.array(FPDatas).astype('float64')

wavelet = 'db4'

# Perform the wavelet decomposition
coeffs = pywt.wavedec2(FPData, wavelet)

# Threshold the coefficients (using hard thresholding)
threshold = 0.1# adjust this threshold to control the amount of noise removal
coeffs_thresh = [pywt.threshold(c, threshold, 'soft') for c in coeffs]

# Reconstruct the signal using the inverse wavelet transform
FPData_Decompos = pywt.waverec2(coeffs_thresh, wavelet)

plt.figure(figsize=(15,6))
    # plt.figure()
plt.plot(FPData[:,1],color='red')
plt.plot(FPData_Decompos[:,1], markerfacecolor='none',color='green')
plt.legend(['Real Data', 'Denoising Data'], loc='best')
plt.show()

Results:

I've adjust the threshold but still same.the denoised data not smoothed.

link of dataset : https://drive.google.com/file/d/1hV0kWe_C0XUZWY-M6hh8UC8RtO9Q521b/view?usp=sharing

expeted result same as the black point below:

Second Case:

SmartInsole = np.array([[  0.,   0.,  79.,  90.,  64.,   3.,   0.,   0.,   0.,   0.,  19.,
        113., 109.,   1.,  25.,   0.,   0.,   0.,   0.,   0.,   0.,  90.,
         99.,  73.,  35.,   0.,   0.,   0.,   0.,  46., 106., 113., 105.,
         52.,   0.,   0.,   0.,   0.,   0.,   0.,  61.,  71.,  53.,  30.,
          0.,  23.,   2.,  11.,  42., 114., 112., 100.,   0.,   0.,   0.,
          0.,   0.,   0.,   4.,   1.,   0.,   0.,   0.,  42.,  47.,  80.,
         86., 125., 121., 111.,  16.,   0.,   0.,   0.,  47.,  72., 112.,
        123., 129.,  82.,   0.,   0.,   0.,  87.,  80.,   0.,   0.,   5.,
          0.],
       [  0.,   0.,  79.,  90.,  64.,   3.,   0.,   0.,   0.,   0.,  19.,
        113., 109.,   1.,  25.,   0.,   0.,   0.,   0.,   0.,   0.,  90.,
         99.,  73.,  35.,   0.,   0.,   0.,   0.,  46., 106., 113., 105.,
         52.,   0.,   0.,   0.,   0.,   0.,   0.,  57.,  68.,  47.,  20.,
          0.,  17.,   1.,  14.,  48., 120., 118., 105.,   0.,   0.,   0.,
          0.,   0.,   0.,   4.,   1.,   0.,   0.,   0.,  42.,  47.,  80.,
         86., 125., 121., 111.,  16.,   0.,   0.,   0.,  47.,  72., 112.,
        123., 129.,  82.,   0.,   0.,   0.,  87.,  80.,   0.,   0.,   5.,
          0.]])

How to implemet above technique to remove noise if the SmartInsole data like above.

SmartInsole Plot:

enter link description here

Answer 1

I think that your problem comes from the fact that you are using wavedec2, which is a 2D Discrete Wavelet Transform, in order to process a 1D array.

Why not use wavedec instead?

Here is an example using scipy's ECG data:

from scipy.misc import electrocardiogram
import matplotlib.pyplot as plt
import pywt
import numpy as np
ecg = electrocardiogram()


FPData = ecg.reshape(10800,-1)
DWTcoeffs = pywt.wavedec(FPData[:,1], 'db4')
DWTcoeffs[-1] = np.zeros_like(DWTcoeffs[-1])
DWTcoeffs[-2] = np.zeros_like(DWTcoeffs[-2])
DWTcoeffs[-3] = np.zeros_like(DWTcoeffs[-3])
DWTcoeffs[-4] = np.zeros_like(DWTcoeffs[-4])
DWTcoeffs[-5] = np.zeros_like(DWTcoeffs[-5])
DWTcoeffs[-6] = np.zeros_like(DWTcoeffs[-6])
DWTcoeffs[-7] = np.zeros_like(DWTcoeffs[-7])
DWTcoeffs[-8] = np.zeros_like(DWTcoeffs[-8])
DWTcoeffs[-9] = np.zeros_like(DWTcoeffs[-9])

filtered_data_dwt=pywt.waverec(DWTcoeffs,'db4',mode='symmetric',axis=-1)
plt.figure(figsize=(15,6))
plt.plot(FPData[:,1],color='red')
plt.plot(filtered_data_dwt, markerfacecolor='none',color='black')
plt.legend(['Real Data', 'Denoised Data'], loc='best')
plt.show()

This returns:

EDIT: In your comment below, you mention that the output (the black curve) looks to smooth. You can get a more detailed output by not zeroing out some of the coefficients. For instance, if you comment out lines:

#DWTcoeffs[-8] = np.zeros_like(DWTcoeffs[-8])
#DWTcoeffs[-9] = np.zeros_like(DWTcoeffs[-9])

Then you get: