How to detect a sudden change in a time series plot in Pandas

Question

I am trying to "detect" a sudden drop in velocity in a series and I'm not sure how to capture it. The details and code are below:

This is a snippet of the Series that I have along with the code to produce it:

velocity_df.velocity.car1

Index   velocity
200     17.9941
201     17.9941
202     18.4031
203     18.4031

Here is a plot of the entire series

I'm trying to detect the sudden drop from 220 to 230-40 and save that out as a Series that looks like this:

Index   velocity
220      14.927
221      14.927
222      14.927
223      14.927
224      14.518
225      14.518
226     16.1538
227     12.2687
228     9.20155
229     6.33885
230     4.49854

I'm just trying to capture an approximate range when there is a sudden decrease in speed so as to use other features.

If I can add any additional information, please let me know. Thank you!

Answer 1

This would be a simple approach, if you want to compare two values one by one:

Given the series from your question, called s you can construct the absolute discrete derivative of your data by subtracting it with a shift of 1:

d = pd.Series(s.values[1:] - s.values[:-1], index=s.index[:-1]).abs()

If you now take the maximum m of that series of absolute differences, you can multiply it with a factor a between 0 and 1 as a threshold:

a = .7
m = d.max()
print(d > m * a)

The last line outputs the indices of the matches.

Building up on this, you could use a sliding window technique such as kernel density estimation, or Parzen window to create more smooth results:

r = d.rolling(3, min_periods=1, win_type='parzen').sum()
n = r.max()

Like before we can print out the matching elements

print(r > n * a)

Which gives the following output

Index
220    False
221    False
222    False
223    False
224    False
225    False
226    False
227     True
228     True
229     True
dtype: bool