CODEWITHSUNDEEP
rmissing-dataimputationimputets
s1368647
s1368647

Reputation: 61

Impute missing values with ROLLING mean in R

I am new to R and struggling with a problem.

I need a function to impute the missing values in a vector according to the mean value of the elements within a window of a given size.

However, this window will move because, say my NA is in position 30, and my window size is 10, the mean should be computed for x[20:40]. So for each found NA, the window-mean will be different.

I have been trying this:

impute.to.window.mean <- function(x, window) {

  na.idx <- is.na(x)  #find missing values in x

  for (na in na.idx) {

    y <- (x[na]-window):(x[na]+window)
    na.idx[na] <- mean(y, na.rm = TRUE)
  }

  return(x)
}

but it is not correct and I don't know how to continue.

Upvotes: 6

Views: 3663

Answers (5)

Raad
Raad

Reputation: 2715

You might want to consider using the imputeTS package. Here is an example of filling in values with a simple moving average and a window of 4:

x <- rnorm(100)
x[c(7, 21, 33)] <- NA

imputeTS::na_ma(x, k = 4, weighting = "simple")

Upvotes: 3

Faustin Gashakamba
Faustin Gashakamba

Reputation: 171

The "Caret" package's preProcess function has a method called "knnImpute" that does exactly that. Give it a go.

Upvotes: 0

G. Grothendieck
G. Grothendieck

Reputation: 270268

Using zoo::rollapply this can be done in one statement. We have used a window of length 5 (2 on either side of the current point) for this example:

library(zoo)

x <- replace(1:20, c(4, 6, 10, 15), NA) # test data


rollapply(c(NA, NA, x, NA, NA), 5, 
    function(x) if (is.na(x[3])) mean(x, na.rm = TRUE) else x[3])

giving:

 [1]  1.000000  2.000000  3.000000  3.333333  5.000000  6.666667  7.000000
 [8]  8.000000  9.000000 10.000000 11.000000 12.000000 13.000000 14.000000
[15] 15.000000 16.000000 17.000000 18.000000 19.000000 20.000000

Upvotes: 2

rawr
rawr

Reputation: 20811

Your indexing is a little off

impute.to.window.mean <- function(x, window) {
  na.idx <- which(is.na(x))  #find missing values in x

  for (na in na.idx) {
    y <- sort(x[(na - window):(na + window)])
    x[na] <- mean(y)
  }

  return(x)
}

Walk through an example

set.seed(1)
x <- sample(10)
na <- 6
x[na] <- NA
# [1]  3  4  5  7  2 NA  9  6 10  1

window <- 3L

I used sort because it drops the NAs in one step; you want the mean of this vector which are all the values that fall in window

sort(x[(na - window):(na + window)])
# [1]  2  5  6  7  9 10

mean(sort(x[(na - window):(na + window)]))
# [1] 6.5

Test your function now

impute.to.window.mean(x, window)
# [1]  3.0  4.0  5.0  7.0  2.0  6.5  9.0  6.0 10.0  1.0

Edit

Actually, you should probably use

y <- sort(x[pmax(1L, (na - window)):pmin(length(x), (na + window))])

instead for the case that an NA occurs at, say, 2, and your window is > 1

## current version
impute.to.window.mean(x, 10)
# Error in x[(na - window):(na + window)] : 
#   only 0's may be mixed with negative subscripts

## version with pmax/pmin
impute.to.window.mean(x, 10)
# [1]  3.000000  4.000000  5.000000  7.000000  2.000000  5.222222  9.000000  6.000000 10.00000 1.000000

mean(sort(x))
# [1] 5.222222

impute.to.window.mean <- function(x, window) {
  na.idx <- which(is.na(x))  #find missing values in x

  for (na in na.idx) {
    # y <- sort(x[(na - window):(na + window)])
    y <- sort(x[pmax(1L, (na - window)):pmin(length(x), (na + window))])
    x[na] <- mean(y)
  }

  return(x)
}

Upvotes: 0

Wietze314
Wietze314

Reputation: 6020

with R base: 

df <- data.frame(x = sample(c(1:10,NA),1000, replace = T))
window <- 10

lapply(1:(nrow(df)-window), function(x) ifelse(is.na(df[x,'x']),mean(df[x:(x+10),'x'],na.rm=T),df[x,'x']))

Only difference I have that I now look forward for the values. But you can alter that to your specifications.

Upvotes: 0

Related Questions