CODEWITHSUNDEEP
rdplyr
Dunois
Dunois

Reputation: 1843

Calculating standard deviation across rows

Say I have the following data:

colA <- c("SampA", "SampB", "SampC")
colB <- c(21, 20, 30)
colC <- c(15, 14, 12)
colD <- c(10, 22, 18)
df <- data.frame(colA, colB, colC, colD)
df
#    colA colB colC colD
# 1 SampA   21   15   10
# 2 SampB   20   14   22
# 3 SampC   30   12   18

I want to get the row means and standard deviations for the values in columns B-D.

I can calculate the rowMeans as follows:

library(dplyr)
df %>% select(., matches("colB|colC|colD")) %>% mutate(rmeans = rowMeans(.))
#   colB colC colD   rmeans
# 1   21   15   10 15.33333
# 2   20   14   22 18.66667
# 3   30   12   18 20.00000

But when I try to calculate the standard deviation using sd(), it throws up an error.

df %>% select(., matches("colB|colC|colD")) %>% mutate(rsds = sapply(., sd(.)))
Error in is.data.frame(x) : 
  (list) object cannot be coerced to type 'double'

So my question is: how do I calculate the standard deviations here?

Edit: I tried sapply() with sd() having read the first answer here.

Additional edit: not necessarily looking for a 'tidy' solution (base R also works just fine).

Upvotes: 11

Views: 9975

Answers (7)

D. Woods
D. Woods

Reputation: 3073

I'm not sure how old/new dplyr's c_across functionality is relative to the prior answers on this page, but here's a solution that is almost directly cut and pasted from the documentation for dplyr::c_across:

df %>% 
  rowwise() %>% 
  mutate(
     mean = mean(c_across(colB:colD)),
     sd = sd(c_across(colB:colD))
  )

# A tibble: 3 x 6
# Rowwise: 
  colA   colB  colC  colD  mean    sd
  <fct> <dbl> <dbl> <dbl> <dbl> <dbl>
1 SampA    21    15    10  15.3  5.51
2 SampB    20    14    22  18.7  4.16
3 SampC    30    12    18  20    9.17

Upvotes: 6

tmfmnk
tmfmnk

Reputation: 39858

A different dplyr and tidyr approach could be:

df %>% 
 pivot_longer(-1) %>%
 group_by(colA) %>%
 mutate(rsds = sd(value)) %>%
 pivot_wider(names_from = "name",
             values_from = "value")

  colA   rsds  colB  colC  colD
  <fct> <dbl> <dbl> <dbl> <dbl>
1 SampA  5.51    21    15    10
2 SampB  4.16    20    14    22
3 SampC  9.17    30    12    18

Or alternatively, using rowwise() and do():

 df %>% 
 rowwise() %>%
 do(data.frame(., rsds = sd(unlist(.[2:length(.)]))))

  colA   colB  colC  colD  rsds
* <fct> <dbl> <dbl> <dbl> <dbl>
1 SampA    21    15    10  5.51
2 SampB    20    14    22  4.16
3 SampC    30    12    18  9.17

Or an option since dplyr 1.0.0:

df %>% 
 rowwise() %>%
 mutate(rsds = sd(c_across(-1)))

Upvotes: 4

randr
randr

Reputation: 314

I see this post is a bit old, but there are some pretty complicated answers so I thought I'd suggest an easier (and faster) approach.

Calculating means of rows is trivial, just use rowMeans:

rowMeans(df[, c('colB', 'colC', 'colD')])

This is vectorised and very fast.

There is no 'rowSd' function, but it is not hard to write one. Here is my 'rowVars' that I use.

rowVars <- function(x, na.rm=F) {
    # Vectorised version of variance filter
    rowSums((x - rowMeans(x, na.rm=na.rm))^2, na.rm=na.rm) / (ncol(x) - 1)
}

To calculate sd:

sqrt(rowVars(df[, c('colB', 'colC', 'colD')]))

Again, vectorised and fast which can be important if the input matrix is large.

Upvotes: 3

moodymudskipper
moodymudskipper

Reputation: 47310

You can use pmap, or rowwise (or group by colA) along with mutate :

library(tidyverse)
df %>% mutate(sd = pmap(.[-1], ~sd(c(...)))) # same as transform(df, sd = apply(df[-1],1,sd))
#>    colA colB colC colD       sd
#> 1 SampA   21   15   10 5.507571
#> 2 SampB   20   14   22 4.163332
#> 3 SampC   30   12   18 9.165151

df %>% rowwise() %>% mutate(sd = sd(c(colB,colC,colD)))
#> Source: local data frame [3 x 5]
#> Groups: <by row>
#> 
#> # A tibble: 3 x 5
#>   colA   colB  colC  colD    sd
#>   <fct> <dbl> <dbl> <dbl> <dbl>
#> 1 SampA    21    15    10  5.51
#> 2 SampB    20    14    22  4.16
#> 3 SampC    30    12    18  9.17

df %>% group_by(colA) %>% mutate(sd = sd(c(colB,colC,colD)))
#> # A tibble: 3 x 5
#> # Groups:   colA [3]
#>   colA   colB  colC  colD    sd
#>   <fct> <dbl> <dbl> <dbl> <dbl>
#> 1 SampA    21    15    10  5.51
#> 2 SampB    20    14    22  4.16
#> 3 SampC    30    12    18  9.17

Upvotes: 3

akrun
akrun

Reputation: 887108

Here is another way using pmap to get the rowwise mean and sd

library(purrr)
library(dplyr)
library(tidur_
f1 <- function(x) tibble(Mean = mean(x), SD = sd(x))
df %>% 
  # select the numeric columns
  select_if(is.numeric) %>%
  # apply the f1 rowwise to get the mean and sd in transmute
  transmute(out = pmap(.,  ~ f1(c(...)))) %>% 
  # unnest the list column
  unnest %>%
  # bind with the original dataset
  bind_cols(df, .)
#   colA colB colC colD     Mean       SD
#1 SampA   21   15   10 15.33333 5.507571
#2 SampB   20   14   22 18.66667 4.163332
#3 SampC   30   12   18 20.00000 9.165151

Upvotes: 3

Hector Haffenden
Hector Haffenden

Reputation: 1369

Try this (using), withrowSds from the matrixStats package,

library(dplyr)
library(matrixStats)

columns <- c('colB', 'colC', 'colD')

df %>% 
  mutate(Mean= rowMeans(.[columns]), stdev=rowSds(as.matrix(.[columns])))

Returns 

   colA colB colC colD     Mean    stdev
1 SampA   21   15   10 15.33333 5.507571
2 SampB   20   14   22 18.66667 4.163332
3 SampC   30   12   18 20.00000 9.165151

Your data

colA <- c("SampA", "SampB", "SampC")
colB <- c(21, 20, 30)
colC <- c(15, 14, 12)
colD <- c(10, 22, 18)
df <- data.frame(colA, colB, colC, colD)
df

Upvotes: 5

Rui Barradas
Rui Barradas

Reputation: 76402

Package magrittr pipes %>% are not a good way to process by rows.
Maybe the following is what you want.

df %>% 
  select(-colA) %>%
  t() %>% as.data.frame() %>%
  summarise_all(sd)
#        V1       V2       V3
#1 5.507571 4.163332 9.165151

Upvotes: 2

Related Questions