CODEWITHSUNDEEP
pythonpandassmoothing
HeadOverFeet
HeadOverFeet

Reputation: 788

Smoothing / noise filtering data in Python

I have table with data as follows

article price   wished outcome
horse    10         10
duck     15         15
child    9        15 - 21
panda    21         21
lamb     24         22
gorilla  23         23

I want to smooth column Price to the wished Price and then put it into dataframe, so that I see the values.

Desired output

Please, is there some built in library - method that smoothens the data? in this format?

I found savitzky-golay filter, moving average, etc. But I fail to make it on these kind of data - where x axis is some product = not value.

Please, can you help?

Thanks!!!

 d = {'Price': [10, 15, 9, 21,24,23], 'Animal': ['horse', 'lamb', 'gorilla', 'child','panda','duck']}
 df = pd.DataFrame(d)

 import matplotlib.pyplot as plt 
 from scipy.optimize import curve_fit
 from scipy.interpolate import interp1d
 from scipy.signal import savgol_filter
 import numpy as np

 x = np.arange(1,len(df)+1)
 y = df['Price']

 xx = np.linspace(x.min(),x.max(), 1001)

 # interpolate + smooth
 itp = interp1d(x,y, kind='quadratic') #kind = 'linear', 'nearest' (dobre      vysledky), slinear (taky ok), cubic (nebrat), quadratic - nebrat
 window_size, poly_order = 1001, 1
 yy_sg = savgol_filter(itp(xx), window_size, poly_order)


 # or fit to a global function
 # to stejne jako scipy.optimize.curve.fit 
 def func(x, A, B, x0, sigma):
     return A+B*np.tanh((x-x0)/sigma)

 fit, _ = curve_fit(func, x, y)
 yy_fit = func(xx, *fit)

 fig, ax = plt.subplots(figsize=(7, 4))
 ax.plot(x, y, 'r.', label= 'Unsmoothed curve')
 ax.plot(xx, yy_fit, 'b--', label=r"$f(x) = A + B \tanh\left(\frac{x-x_0}     {\sigma}\right)$")
 ax.plot(xx, yy_sg, 'k', label= "Smoothed curve")
 plt.legend(loc='best')


 I am getting : AttributeError: 'range' object has no attribute 'min'

Savitzky golay is producing very strange values. With window lenght 1000

enter image description here

When I set window to len(df) +1 (in order it to be odd) then I get these data:

enter image description here

Upvotes: 0

Views: 2392

Answers (1)

ignoring_gravity
ignoring_gravity

Reputation: 10531

You're getting that error because of the following line: x = range(1,len(df)).

As the error tells you, a range object has no attribute min.

However, numpy.array()s do, so if you change that line to x = np.arange(1, len(df)) then this error (at least) will disappear.

EDIT: In order for the function to do what you want it to do, you should change it to x = np.arange(1, len(df)+1)

Upvotes: 1

Related Questions