CODEWITHSUNDEEP
rduplicates
Stephen Turner
Stephen Turner

Reputation: 2604

Remove duplicates keeping entry with largest absolute value

Let's say I have four samples: id=1, 2, 3, and 4, with one or more measurements on each of those samples:

> a <- data.frame(id=c(1,1,2,2,3,4), value=c(1,2,3,-4,-5,6))
> a
  id value
1  1     1
2  1     2
3  2     3
4  2    -4
5  3    -5
6  4     6

I want to remove duplicates, keeping only one entry per ID - the one having the largest absolute value of the "value" column. I.e., this is what I want:

> a[c(2,4,5,6), ]
  id value
2  1     2
4  2    -4
5  3    -5
6  4     6

How might I do this in R?

Upvotes: 39

Views: 32500

Answers (7)

IRTFM
IRTFM

Reputation: 263332

First. Sort in the order putting the less desired items last within id groups

 aa <- a[order(a$id, -abs(a$value) ), ] #sort by id and reverse of abs(value)

Then: Remove items after the first within id groups

aa[ !duplicated(aa$id), ] #logical index extracts only first row within each id
  id value
2  1     2
4  2    -4
5  3    -5
6  4     6

This could keep the minimum if the sort were not reversed.

Upvotes: 55

dmca
dmca

Reputation: 685

You can do this with dplyr as follows:

library(dplyr)
a %>%
  group_by(name) %>%
  filter(n == max(n)) %>%
  ungroup()

Upvotes: 1

nghauran
nghauran

Reputation: 6768

Here is a dplyr approach

library(dplyr)
a %>% 
        group_by(id) %>%
        top_n(1, abs(value))

# A tibble: 4 x 2
# Groups:   id [4]
#     id value
#  <dbl> <dbl>
#1     1     2
#2     2    -4
#3     3    -5
#4     4     6

Upvotes: 15

nograpes
nograpes

Reputation: 18323

Check out ?aggregate:

aggregate(value~id,a,function(x) x[which.max(abs(x))])

I like the answer by @DWin, but I would like show how this could also work with metadata:

aa<-merge(aggregate(value~id,a,function(x) x[which.max(abs(x))]),a)
# Fails if the max value is duplicated for a single id without next line.
aa[!duplicated(aa),]

I couldn't help myself and created one last answer:

do.call(rbind,lapply(split(a,a$id),function(x) x[which.max(abs(x$value)),]))

Upvotes: 10

BenBarnes
BenBarnes

Reputation: 19454

A data.table approach might be in order if your data set is very large:

library(data.table)

aDT <- as.data.table(a)
setkey(aDT,"id")

aDT[J(unique(id)), list(value = value[which.max(abs(value))])]


Or a not as fast, but still fast, alternative :

library(data.table)
as.data.table(a)[, .SD[which.max(abs(value))], by=id]

This version returns all the columns of a, in case there are more in the real dataset.

Upvotes: 15

A5C1D2H2I1M1N2O1R2T1
A5C1D2H2I1M1N2O1R2T1

Reputation: 193517

Another approach (though the code might look a little cumbersome) is to use ave():

a[which(abs(a$value) == ave(a$value, a$id, 
                            FUN=function(x) max(abs(x)))), ]
#   id value
# 2  1     2
# 4  2    -4
# 5  3    -5
# 6  4     6

Upvotes: 5

Maiasaura
Maiasaura

Reputation: 32986

library(plyr)
ddply(a, .(id), function(x) return(x[which(abs(x$value)==max(abs(x$value))),]))

Upvotes: 3

Related Questions