CODEWITHSUNDEEP
rfor-loopdataframemeanstandard-deviation
Anagha
Anagha

Reputation: 3699

Calculate Mean, SD, and replace values in R

I have the above data frame, Date&Time with corresponding signal value.

  1. I need to replace all the positive value with 0
  2. Once replaced, for every 60 seconds, I need to calculate mean and Std dev and replace the value with mean which deviating a lot.

For example, for the first 60 seconds, if the value at 2017-08-23 07:49:58 is deviating more from SD, then it should be replaced by mean. That means "59" should be replaced by mean

     date-time             RSSI
    2017-08-23 07:49:38    -68
    2017-08-23 07:49:48    -69
    2017-08-23 07:49:58    -59
    2017-08-23 07:50:08    -65
    2017-08-23 07:50:18     127
    2017-08-23 07:50:28    -74
    2017-08-23 07:50:38     127
    2017-08-23 07:50:48    -74
    2017-08-23 07:50:58     127
    2017-08-23 07:51:08    -74
    2017-08-23 07:51:18    -65
    2017-08-23 07:51:28     127
    2017-08-23 07:51:38    -59
    2017-08-23 07:51:48    -62
    2017-08-23 07:51:58    -57

Expected output:

Output 1:

  date-time              RSSI
  2017-08-23 07:49:38   -68
  2017-08-23 07:49:48   -69
  2017-08-23 07:49:58   -59
  2017-08-23 07:50:08   -65
  2017-08-23 07:50:18    0

Output 2:

  date-time              RSSI
  2017-08-23 07:49:38   -68
  2017-08-23 07:49:48   -69
  2017-08-23 07:49:58   **-62**
  2017-08-23 07:50:08   -65
  2017-08-23 07:50:18   **-62**

Here -62 is mean and its replaced

Upvotes: 0

Views: 528

Answers (1)

mathdotrandom
mathdotrandom

Reputation: 121

Don't use for loops in R. Try and use vectored solutions and if you need performance usually the package data.table is what you want.

library(data.table)
dt = data.table("date-time"=c(as.POSIXct(c("2017-08-23 07:49:38", "2017-08-23 07:49:48", "2017-08-23 07:49:58", "2017-08-23 07:50:08", "2017-08-23 07:50:18", "2017-08-23 07:50:28" ))), RSSI=c(-68, -69, -59, -65, 127, -74))

dt[RSSI > 0 , RSSI:=NA] #replacing positive ones with NA
print(dt)
dt[ , minute:=floor(as.numeric(`date-time`)/60)] # calculate for each time in which minute it belongs
# calculate mean and standard deviation per group
dt[ , c("mean", "stdev") := list(mean(RSSI, na.rm=TRUE), sd(RSSI, na.rm=TRUE)), by = minute] #ignoring the NA outliers
dt[ abs(RSSI - mean) > stdev  | is.na(RSSI), RSSI:=round(mean)] #round should return an integer
print(dt)

The solution you want should look similar to this. Reading a csv with data.table works best with the function fread.

Upvotes: 2

Related Questions