numpy polyfit yields nonsense

Question

I am trying to fit these values:

This is my code:

  for i in range(-area,area):
    stDev1= []
    for j in range(-area,area):
        stDev0 = stDev[i+i0][j+j0]
        stDev1.append(stDev0)
    slices[i] = stDev1
fitV = []
xV = []

for l in range(-area,area):
    y = np.asarray(slices[l])
    x =  np.arange(0,2*area,1)

for m in range(-area,area):
        fitV.append(slices[m][l])
        xV.append(l)


fit = np.polyfit(xV,fitV,4)
yfit = function(fit,area)

x100 = np.arange(0,100,1)

plt.plot(xV,fitV,'.')
plt.savefig("fits1.png")

def function(fit,area):
   yfit = []
   for x in range(-area,area):
       yfit.append(fit[0]+fit[1]*x+fit[2]*x**2+fit[3]*x**3+fit[4]*x**4)
   return(yfit)

i0 = 400
j0 = 400
area = 50 
stdev = 2d np.array([1300][800]) #just an image of "noise" feel free to add any image // 2d np array you like. 

This yields:

obviously this is completly wrong? I assume I miss understand the concept of polyfit? From the doc the requirement is that I feed it with with two arrays of shape x[i] y[i]? My values in

  xV = [ x_1_-50,x_1_-49,...,x_1_49,x_2_-50,...,x_49_49] 

and my ys are:

  fitV = [y_1_-50,y_1_-49,...,y_1_49,...y_2_-50,...,y_2_49]

Answer 1

I do not completely understand your program. In the future, it would be helpful if you were to distill your issue to a MCVE. But here are some thoughts:

1. It seems, in your data, that for a given value of x there are multiple values of y. Given (x,y) data, polyfit returns a tuple that represents a polynomial function, but no function can map a single value of x onto multiple values of y. As a first step, consider collapsing each set of y values into a single representative value using, for example, the mean, median, or mode. Or perhaps, in your domain, there's a more natural way to do this.

2. Second, there is an idiomatic way to use the pair of functions np.polyfit and np.polyval, and you're not using them in the standard way. Of course, numerous useful departures from this pattern exist, but first make sure you understand the basic pattern of these two functions.

   a. Given your measurements y_data, taken at times or locations x_data, plot them and make a guess as to the order of the fit. That is, does it look like a line? Like a parabola? Let's assume you believe your data to be parabolic, and that you'll use a second order polynomial fit.

   b. Make sure that your arrays are sorted in order of increasing x. There are many ways to do this, but np.argsort is a easy one.

   c. Run polyfit: p = polyfit(x_data,y_data,2), which returns a tuple containing the 2nd, 1st, and 0th order coefficients in p, (c2,c1,c0).

   d. In the idiomatic use of polyfit and polyval, next you would generate your fit: polyval(p,x_data). Or perhaps you want the fit to be sampled more coarsely or finely, in which case you might take a subset of x_data or interpolate more values in x_data.

A complete example is below.

import numpy as np
from matplotlib import pyplot as plt

# these are your measurements, unsorted
x_data = np.array([18, 6, 9, 12 , 3, 0, 15])
y_data = np.array([583.26347805, 63.16059915, 100.94286909, 183.72581827, 62.24497418,
                   134.99558191, 368.78421529])

# first, sort both vectors in increasing-x order:
sorted_indices = np.argsort(x_data)
x_data = x_data[sorted_indices]
y_data = y_data[sorted_indices]

# now, plot and observe the parabolic shape:
plt.plot(x_data,y_data,'ks')
plt.show()

# generate the 2nd order fitting polynomial:
p = np.polyfit(x_data,y_data,2)

# make a more finely sampled x_fit vector with, for example
# 1024 equally spaced points between the first and last
# values of x_data
x_fit = np.linspace(x_data[0],x_data[-1],1024)

# now, compute the fit using your polynomial:
y_fit = np.polyval(p,x_fit)

# and plot them together:
plt.plot(x_data,y_data,'ks')
plt.plot(x_fit,y_fit,'b--')
plt.show()

Hope that helps.