CODEWITHSUNDEEP
rdataframeoutliers
Neuls
Neuls

Reputation: 123

R Remove outliers in a dataframe grouped by factor

I have a dataframe with measurements of 3 parameters grouped by sample:

ORD        curv   exp   rep         mu           lam       abs
1  Combi pH=7 Curva_F_Cor Exp_F  Rep1 0.15637365   714.947.305 0.4990000
2  Combi pH=7 Curva_F_Cor Exp_F Rep10 0.12817901 6.797.925.883 0.4914276
3  Combi pH=7 Curva_F_Cor Exp_F Rep11 0.13392221 6.765.638.528 0.5261217
4  Combi pH=7 Curva_F_Cor Exp_F  Rep2 0.09683254 6.671.151.868 0.4236507
5  Combi pH=7 Curva_F_Cor Exp_F  Rep3 0.11249738 6.868.057.298 0.4899013
6  Combi pH=7 Curva_F_Cor Exp_F  Rep4 0.10878719 6.829.856.006 0.4876704
7  Combi pH=7 Curva_F_Cor Exp_F  Rep5 0.11019295 6.758.654.665 0.4871269
8  Combi pH=7 Curva_F_Cor Exp_F  Rep6 0.12100511 6.733.007.508 0.4923079
9  Combi pH=7 Curva_F_Cor Exp_F  Rep7 0.09803942 6.791.743.116 0.4185484
10 Combi pH=7 Curva_F_Cor Exp_F  Rep8 0.13842086 6.909.115.228 0.5392007
11 Combi pH=7 Curva_F_Cor Exp_F  Rep9 0.12778964 6.779.856.345 0.5475924
12    ORD0793 Curva_F_Cor Exp_F  Rep1 0.13910441 7.051.072.489 0.4706000
13    ORD0793 Curva_F_Cor Exp_F  Rep2 0.12603702 7.143.108.903 0.4436000
14    ORD0793 Curva_F_Cor Exp_F  Rep3 0.12670842 6.989.806.663 0.4258000
15    ORD0795 Curva_F_Cor Exp_F  Rep1 0.12982122 7.029.434.508 0.4996000
16    ORD0795 Curva_F_Cor Exp_F  Rep2 0.13648100 6.776.386.442 0.4896000
17    ORD0795 Curva_F_Cor Exp_F  Rep3 0.13593685 7.161.375.293 0.4766000
18    ORD0799 Curva_F_Cor Exp_F  Rep1 0.13906691 7.065.198.206 0.4806000
19    ORD0799 Curva_F_Cor Exp_F  Rep2 0.14822216    70.824.584 0.4640000
20    ORD0799 Curva_F_Cor Exp_F  Rep3 0.10630870 6.669.130.811 0.4686809
21    ORD0839 Curva_F_Cor Exp_F  Rep1 0.16717843 6.133.730.567 0.5458000
22    ORD0839 Curva_F_Cor Exp_F  Rep2 0.09995048 7.119.564.022 0.4026000
23    ORD0839 Curva_F_Cor Exp_F  Rep3 0.15911022 7.321.225.246 0.5118000
24    ORD0843 Curva_F_Cor Exp_F  Rep1 0.12508123 6.579.839.732 0.5458217
25    ORD0843 Curva_F_Cor Exp_F  Rep2 0.16396603 6.536.282.149 0.5210000
26    ORD0843 Curva_F_Cor Exp_F  Rep3 0.15029945 7.015.299.122 0.4838000
27    ORD0847 Curva_F_Cor Exp_F  Rep1 0.11697558 7.076.730.379 0.4148000
28    ORD0847 Curva_F_Cor Exp_F  Rep2 0.15276497 7.181.749.575 0.5088000
29    ORD0847 Curva_F_Cor Exp_F  Rep3 0.15533901   710.518.294 0.5348000
30    ORD0856 Curva_F_Cor Exp_F  Rep1 0.11217122 7.940.648.197 0.4130000
31    ORD0856 Curva_F_Cor Exp_F  Rep2 0.12010424 8.359.758.086 0.4446000
32    ORD0856 Curva_F_Cor Exp_F  Rep3 0.13337373   811.057.251 0.4780000

I would like to remove outliers of mu lam and abs of each sample contained in ORD column.

I found in this forum a function to remove outliers:

remove_outliers <- function(x, na.rm = TRUE, ...) {
  qnt <- quantile(x, probs=c(.25, .75), na.rm = na.rm, ...)
  H <- 1.5 * IQR(x, na.rm = na.rm)
  y <- x
  y[x < (qnt[1] - H)] <- NA
  y[x > (qnt[2] + H)] <- NA
  y
}

but I just know how to apply them in a numeric vector or using lapply to apply the function to each column of the data frame, but I have no clue how to apply tje function to the data frame grouping by sample. Something like remove_outliers(mu~ORD, data=df, na.rm=TRUE)

I'd thank any help

Upvotes: 4

Views: 4774

Answers (2)

Parfait
Parfait

Reputation: 107767

Alternatively, in base R consider by which creates a list of dataframes sliced by listed factors, df$ORD. Afterwards, row bind all df elements into one compiled dataframe. And use an sapply to process function on numeric columns:

dflist <- by(df, df$ORD, function(i){
  i[c("mu","lam","abs")] <- sapply(i[c("mu","lam","abs")], remove_outliers)
  return(i)
})

newdf <- do.call(rbind, dflist)
rownames(newdf) <- NULL

Upvotes: 2

www
www

Reputation: 39174

We can use functions from dplyr to achieve this. You may want to group_by the column as groups, and the use mutate to update your columns.

library(dplyr)

You can apply only one column by specifying the column name and the function as follows.

# Apply the finction to one column
dt2 <- dt %>%
  group_by(ORD) %>%
  mutate(mu = remove_outliers(mu))

You can also apply this to multiple columns by using mutate_at with specifying multiple column names in vars().

# Apply the function to multiple columns
dt3 <- dt %>%
  group_by(ORD) %>%
  mutate_at(vars(mu, abs), funs(remove_outliers))

Upvotes: 4

Related Questions