CODEWITHSUNDEEP
r
Narjems
Narjems

Reputation: 111

R- sapply for standard deviation returns a column of NAs

This may be very basic but, I'm trying to create a column of standard deviations for a variable Returns_Close_exp. This variable is actually a numeric vector . So what I want is not the standard deviation of the whole vector, but between two elements.

Here is how I created the vector and how it looks like: 

 Returns_Close_exp<-diff(log(Data_new$Close_exp), lag=1)
 Returns_Close_exp<-append(Returns_Close_exp,"",0)
 Returns_Close_exp<-as.numeric(Returns_Close_exp)

Head of the vector:

dput(head(Returns_Close_exp))
c(NA, 0, 0.00121876921624686, -0.00121876921624686, -0.00122025634730871, 
-0.00981602975444584)

What I tried to get the standard deviations is:

vol_close_exp<-sapply(Returns_Close_exp,sd)

But I get a column of NAs. Does anyone know what is wrong and how to correct it? Thank you

Upvotes: 0

Views: 90

Answers (1)

duckmayr
duckmayr

Reputation: 16930

There are a couple of ways you could do this; using sapply() on a vector of indices (essentially as a loop), or using some sort of rolling function, such as with RcppRoll::roll_sd:

vec <- c(NA, 0, 0.00121876921624686, -0.00121876921624686, -0.00122025634730871,
         -0.00981602975444584)
# Solution with base R
sapply(1:(length(vec)-1), function(i) sd(c(vec[i], vec[i+1])))
#> [1]          NA 8.61800e-04 1.72360e-03 1.05156e-06 6.07813e-03
# Solution with RcppRoll (recommended when performance is key)
RcppRoll::roll_sd(vec, 2)
#> [1]          NA 8.61800e-04 1.72360e-03 1.05156e-06 6.07813e-03

What you were doing initially was applying the function sd() to each number in your vector, and the sd() of a single number is always NA.

Update: Benchmarks

I mentioned RcppRoll was recommended when performance is key. Let's see just how much faster the RcppRoll solution is:

vec <- c(NA, 0, 0.00121876921624686, -0.00121876921624686, -0.00122025634730871,
         -0.00981602975444584)
# Benchmarks:
library(microbenchmark)
microbenchmark(base = sapply(1:(length(vec)-1), function(i) sd(c(vec[i], vec[i+1]))),
               rcpproll = RcppRoll::roll_sd(vec, 2))
#> Unit: microseconds
#>      expr      min       lq      mean   median        uq       max neval
#>      base 1042.251 1083.269 1264.2698 1133.120 1287.4990 10036.937   100
#>  rcpproll  124.930  133.654  161.4393  145.947  168.3785   286.695   100
#>  cld
#>    b
#>   a
# Let's benchmark with bigger data:
set.seed(123)
vec <- rnorm(1e4)
microbenchmark(base = sapply(1:(length(vec)-1), function(i) sd(c(vec[i], vec[i+1]))),
               rcpproll = RcppRoll::roll_sd(vec, 2))
#> Unit: milliseconds
#>      expr        min          lq       mean      median          uq
#>      base 1966.86067 2063.439484 2141.24892 2134.337090 2198.671640
#>  rcpproll    3.55177    3.701657    4.01187    3.786363    3.904616
#>         max neval cld
#>  2525.95089   100   b
#>    20.71965   100  a
all.equal(sapply(1:(length(vec)-1), function(i) sd(c(vec[i], vec[i+1]))),
          RcppRoll::roll_sd(vec, 2))
#> [1] TRUE

Upvotes: 3

Related Questions