CODEWITHSUNDEEP
rdataframenesteddplyr
user42485
user42485

Reputation: 811

Filtering values within nested data column using dplyr

I want to filter out outliers (above or below mean +- 3 * sem) in a nested data frame using dplyr. I don't know how to formulate the code to refer to values within the nested $data column below. For example, 

df <- tibble(
  a = rep(c('x', 'y', 'z'),100),
  value = c(1:298, 999, 1000)) %>% 
  nest(value) %>% 
  mutate(mn = map_dbl(data, ~mean(.$value, na.rm = T)),
         sem = map_dbl(data, ~sqrt(var(.$value,na.rm=T)/length(na.omit(.$value)))),
         upper = mn + 3*sem,
         lower = mn - 3*sem)

How do I now filter the 'value' column in the data column based on these upper and lower bounds?

Upvotes: 0

Views: 678

Answers (1)

akuiper
akuiper

Reputation: 214977

You can use Map to loop over data, lower and upper columns in parallel and then filter nested data based on the value column:

df %>% 
    mutate(data = Map(function(data, upper, lower) filter(data, data$value < upper, data$value > lower), data, upper, lower))

# A tibble: 3 x 6
#      a              data    mn       sem    upper    lower
#  <chr>            <list> <dbl>     <dbl>    <dbl>    <dbl>
#1     x <tibble [18 x 1]> 149.5  8.703448 175.6103 123.3897
#2     y <tibble [24 x 1]> 157.5 12.072696 193.7181 121.2819
#3     z <tibble [24 x 1]> 158.5 12.072696 194.7181 122.2819

Upvotes: 1

Related Questions