rebrid
rebrid

Reputation: 440

Python to determine when array is not significantly increasing or decreasing anymore

Given an array like the following:

array([ 33.5 , -12.17,  -1.58,  -6.15,   4.73,  -5.62,  -5.59,  -0.79,
        -4.73,   0.58,   1.07,  -1.33,   1.22,   2.72,  -3.24,  -3.33,
         0.65,   0.79,   2.66,  -1.45,   2.  ,   3.52,  -0.6 ,   1.36,
        -0.51,  -1.25,  -0.37,   0.55,   2.01,  -0.25,   0.08,  -0.38,
         0.94,   0.4 ,  -1.15,  -1.2 ,  -1.11,   1.33,  -1.11,   1.02,
        -0.55,   0.59,  -0.8 ,  -0.28,  -0.61,   0.48,  -1.94,   0.14,
        -0.79,  -1.34,   0.46,   0.69,  -0.08,  -0.2 ,   0.16,  -1.42,
        -0.29,  -0.48,  -0.03,  -0.03,  -0.39,   0.74,   0.04,   0.5 ,
         0.2 ,   0.09,   1.53,  -0.98,   0.22,   0.86,  -0.05,   0.06,
         0.08,  -0.69,  -0.54,  -0.77,   0.35,   0.45,  -0.16,  -0.07,
        -0.32,   1.01,   0.21,  -0.24,   0.06,  -0.54,  -0.2 ,  -0.36,
         0.27,   0.36,   0.16,  -0.29,   0.04,  -0.53,   0.46,   0.37,
         0.14,  -0.41,   0.3 ,  -0.09,  -0.49,  -0.26,  -0.16,  -0.33,
        -0.03,  -0.46,   0.14,   0.01,  -0.41,   0.29,  -0.17,  -0.16,
        -0.09,   0.1 ,   0.04,  -0.08,  -0.33,  -0.06,  -0.09,  -0.21,
         0.06,  -0.31,  -0.23,  -0.15,   0.02,  -0.11,  -0.49,   0.22,
         0.49,  -0.22,   0.07,  -0.02,  -0.07,  -0.47,  -0.22,  -0.13,
         0.22,   0.23,   0.17,  -0.18,  -0.09,  -0.22,  -0.29,   0.19,
         0.01,   0.13,   0.22,  -0.29,   0.01,  -0.11,  -0.33,   0.1 ,
         0.1 ,   0.14,  -0.09,  -0.33,  -0.01,   0.29,   0.07,   0.1 ,
         0.09,   0.24,  -0.07,   0.2 ,  -0.13,  -0.04,  -0.39,  -0.12,
        -0.27,   0.01,   0.12,   0.05,  -0.02,  -0.1 ,   0.21,  -0.13,
        -0.02,  -0.11,  -0.03,  -0.19,   0.16,   0.14,   0.06,  -0.12,
         0.04,  -0.47,  -0.06,   0.2 ,   0.13,   0.11,  -0.17,  -0.14,
        -0.02,  -0.33,  -0.21,  -0.22,  -0.1 ,   2.72,   0.03,  -0.09,
         0.03,  -0.07,  -0.14,  -0.03,  -0.16,   0.03,  -0.08,   0.02,
         0.04,  -0.1 ,  -0.04,  -0.05,   0.13,  -0.39,   0.01,  -0.16,
         0.2 ,   0.13,  -0.24,  -0.27,  -0.08,  -0.62,  -0.04,  -0.31,
         0.02,   0.25,   0.23,   0.05,  -0.12,  -0.02,   0.1 ,  -0.02,
        -0.09,   0.27,  -0.14,   0.15,  -0.14,   0.03,  -0.13,  -0.09,
        -0.12,  -0.07,  -0.1 ,   0.06,  -0.13,  -0.05,  -0.07,  -0.17,
        -0.03,  -0.  ,   0.25,   0.1 ,   0.06,   0.16,  -0.01,   0.28,
         0.  ,  -0.07,   0.19,   0.24,   0.02,   0.09,  -0.05,  -0.06,
        -0.05,   0.08,  -0.01,   0.05,  -0.12,  -0.  ,   0.01,  -0.13,
        -0.1 ,   0.05,  -0.18,  -0.13,   0.3 ,   0.21,   0.17,  -0.09,
         0.21,  -0.02,  -0.04,   0.14,   0.22,  -0.09,  -0.04,   0.08,
         0.15,  -0.08,   0.04,  -0.07,  -0.1 ,   0.01,   0.04,  -0.01,
         0.06,   0.01,   0.27,  -0.02,   0.07,  -0.02,   0.22,   0.11,
        -0.03,   0.16,   0.03,  -0.19,   0.16,   0.05,  -0.08,   0.07,
         0.03,  -0.17,   0.13,   0.13,   0.13,  -0.12,  -0.16,  -0.16,
         0.06,   0.2 ,  -0.07,  -0.1 ,  -0.05,  -0.17,   0.02,   0.13,
        -0.04,  -0.07,  -0.17,  -0.18,   0.03,  -0.17,  -0.07,   0.12,
        -0.05,  -0.  ,  -0.  ,   0.14,  -0.13,   0.03])

I'd like to find the index at which the array is not fluctuating, or say when considering the next item is not significant anymore.

How do I determine this point? Is there a way to compute this with pandas for example? Maybe considering the total average and check if adding an extra item in the computation would increase or decrease significantly the total average?

I tried to find local min and max

 extrema = np.concatenate((argrelmin(array)[0], argrelmax(array)[0]))

but that returns basically the whole array length because there still are fluctuations in the array

Upvotes: 0

Views: 167

Answers (2)

kezzos
kezzos

Reputation: 3221

You could assess this by using a sliding window and looking at the standard deviation over the window. A simple threshold might suffice. For example:

k = 6  # window size
thresh = 0.5
for i in range(0, len(a) - k):
    b = np.std(a[i:i+k])
    if b < thresh:
        print(i)
        break
>>> 55

Upvotes: 1

Thomas Schillaci
Thomas Schillaci

Reputation: 2453

You can do like this, tuning envelope as you wish:

import numpy as np
import matplotlib.pyplot as plot

x = np.array([ 33.5 , ...])

mean = np.mean(x)
envelope = 0.1

for i in range(len(x)-1, -1, -1):
    if abs(x[i]-mean) > envelope:
        i -= 1
        break

plot.plot(x, '.')
plot.axvline(i)
plot.show()

With envelope = 0.5

chart

With envelope = 0.1

chart

Upvotes: 1

Related Questions