CODEWITHSUNDEEP
pythondata-fitting
kb3hts
kb3hts

Reputation: 165

Finding the point of a slope change as a free parameter

Say I have two lists of data as follows:

x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
y = [1, 2, 3, 4, 5, 6, 8, 10, 12, 14]

That is, it's pretty clear that merely fitting a line to this data doesn't work, but instead the slope changed at a point in the data. (Obviously, one can pinpoint from this data set pretty easily where that change is, but it's not as clear in the set I'm working with so let's ignore that.) Something with the derivative, I'm guessing, but the point here is I want to treat this as a free parameter where I say "it's this point, +/- this uncertainty, and here is the linear slope before and after this point."

Note, I can do this with an array if it's easier. Thanks!

Upvotes: 5

Views: 16712

Answers (4)

Ricardo
Ricardo

Reputation: 691

Knee point might be a potential solution.

from kneed import KneeLocator
import numpy as np 
x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y = np.array([1, 2, 3, 4, 5, 6, 8, 10, 12, 14])
kn = KneeLocator(x, y, curve='convex', direction='increasing') 
# You can use array y to automatically determine 'convex' and 'increasing' if y is well-behaved
idx = (np.abs(x - kn.knee)).argmin()

>>> print(x[idx], y[idx])
6 6

Upvotes: 2

Alexander
Alexander

Reputation: 109536

You can calculate the slope as the difference between each pair of points (the first derivative). Then check where the slope changes (the second derivative). If it changes, append the index location to idx, the collection of points where the slope changes.

Note that the first point does not have a unique slope. The second pair of points will give you the slope, but you need the third pair before you can measure the change in slope.

idx = []
prior_slope = float(y[1] - y[0]) / (x[1] - x[0])
for n in range(2, len(x)):  # Start from 3rd pair of points.
    slope = float(y[n] - y[n - 1]) / (x[n] - x[n - 1])
    if slope != prior_slope:
        idx.append(n)
    prior_slope = slope

>>> idx
[6]

Of course this could be done more efficiently in Pandas or Numpy, but I am just giving you a simple Python 2 solution.

A simple conditional list comprehension should also be pretty efficient, although it is more difficult to understand.

idx = [n for n in range(2, len(x)) 
       if float(y[n] - y[n - 1]) / (x[n] - x[n - 1]) 
       != float(y[n - 1] - y[n - 2]) / (x[n - 1] - x[n - 2])]

Upvotes: 2

Dadep
Dadep

Reputation: 2788

I'm not sure to understand very well what you want but you can see the evolution this way (derivative): 

>>> y = [1, 2, 3, 4, 5, 6, 8, 10, 12, 14]
>>> dy=[y[i+1]-y[i] for i in range(len(y)-1)]
>>> dy
[1, 1, 1, 1, 1, 2, 2, 2, 2]

and then find the point where it change (second derivative):

>>> dpy=[dy[i+1]-dy[i] for i in range(len(dy)-1)]
>>> dpy
[0, 0, 0, 0, 1, 0, 0, 0]

if you want the index of this point : 

>>> dpy.index(1)
4

that can give you the value of the last point before change of slope :

>>> change=dpy.index(1)
>>> y[change]
5

In your y = [1, 2, 3, 4, 5, 6, 8, 10, 12, 14] the change happen at the index [4] (list indexing start to 0) and the value of y at this point is 5.

Upvotes: 3

Mahdi
Mahdi

Reputation: 3238

Here is a plot of your data:

enter image description here

You need to find two slopes (== taking two derivatives). First, find the slope between every two points (using numpy):

import numpy as np 
x = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10],dtype=np.float)
y = np.array([1, 2, 3, 4, 5, 6, 8, 10, 12, 14],dtype=np.float)
m = np.diff(y)/np.diff(x)
print (m)
# [ 1.  1.  1.  1.  1.  2.  2.  2.  2.]

Clearly, slope changes from 1 to 2 in the sixth interval (between sixth and seventh points). Then take the derivative of this array, which tells you when the slope changes:

print (np.diff(m))
[ 0.  0.  0.  0.  1.  0.  0.  0.]

To find the index of the non-zero value:

idx = np.nonzero(np.diff(m))[0]
print (idx)
# 4

Since we took one derivative with respect to x, and indices start from zero in Python, idx+2 tells you that the slope is different before and after the sixth point.

Upvotes: 16

Related Questions