CODEWITHSUNDEEP
rfilteringrscripttrend
nando_5g
nando_5g

Reputation: 41

How to filter a trend?

Is it possible to filter a trend like this:

set.seed(1)
n=1000
mu = c(rep(1,100),rep(3,100),rep(5,100),rep(2,100),rep(1,600))
y = mu + rnorm(n)

and then obtain a numerical vector that defines the new trend? It would be optimal if you could also use different threshold values

enter image description here

Upvotes: 0

Views: 219

Answers (1)

Allan Cameron
Allan Cameron

Reputation: 174278

It sounds like you are looking for a smoothing function. There are many ways to achieve this: for example rolling average, loess, generalized additive models. If you want the trend to be composed of straight line sections, as in your example, you could try a regression with b-splines and degree 1.

This little function would perform such a task:

library(splines)

smoother <- function(x, n = floor(length(x) / 10), deg = 1) {
  predict(lm(y ~ bs(seq_along(y), knots = seq(1, length(y), n), degree = deg)))
}

The x argument is the data from which you are trying to find the trend, and n is the number of measurements between knots (that is, the points where the gradient of the line can change). deg is the degree of the polynomial used (1 for straight line segments, and higher numbers for smoother polynomial fits).

Trying this on your example, we would get something like this:

plot(y, type = 'l')

trend <- smoother(y, 50)
lines(trend, col = 'red')

enter image description here

Or if you wanted a less jagged line:

plot(y, type = 'l')
lines(smoother(y, 75, 4), col = 'red')

enter image description here

Upvotes: 4

Related Questions