CODEWITHSUNDEEP
rdata.table
zkurtz
zkurtz

Reputation: 3288

data.table select first row of each group limited to n-1 columns?

The data.table help for the ".SD" function shows how to select the first row of each group:

DT = data.table(x=rep(c("b","a","c"),each=3), v=c(1,1,1,2,2,1,1,2,2), y=c(1,3,6), a=1:9, b=9:1)
DT

DT[, .N, by=x]                         # number of rows in each group

This mostly works well for me, but it breaks when I use all columns to define the group and I don't see why, so I wonder if it's a bug. Example:

# Selecting by n-1 columns works:
DT[, .SD[1], by=c("x", "y", "v", "a")]
   x y v a b
1: b 1 1 1 9
2: b 3 1 2 8
3: b 6 1 3 7
4: a 1 2 4 6
5: a 3 2 5 5
6: a 6 1 6 4
7: c 1 1 7 3
8: c 3 2 8 2
9: c 6 2 9 1

# The result of selecting by all columns is not what I expected:
DT[, .SD[1], by=c("x", "y", "v", "a", "b")]
Empty data.table (0 rows) of 5 cols: x,y,v,a,b

Upvotes: 1

Views: 2460

Answers (1)

akuiper
akuiper

Reputation: 214937

As commented by @christoph, .SD doesn't include group columns (which I believe is for efficiency purpose so as not to store duplicated group values), you can verify it by doing this:

unique(DT[, .(name = names(.SD)), by=c('x','v')]$name)
# [1] "y" "a" "b"

unique(DT[, .(name = names(.SD)), by=c('x','v','a')]$name)
# [1] "y" "b"

So if you group by all columns, .SD has nothing in it; And for your specific case, you can just use unique and pass the group variables to the by parameter, which will drop duplicates based on the by columns:

unique(DT, by=c('x','v'))

#   x v y a b
#1: b 1 1 1 9
#2: a 2 1 4 6
#3: a 1 6 6 4
#4: c 1 1 7 3
#5: c 2 3 8 2

unique(DT, by=c('x','v','y','a','b'))

#   x v y a b
#1: b 1 1 1 9
#2: b 1 3 2 8
#3: b 1 6 3 7
#4: a 2 1 4 6
#5: a 2 3 5 5
#6: a 1 6 6 4
#7: c 1 1 7 3
#8: c 2 3 8 2
#9: c 2 6 9 1

Upvotes: 3

Related Questions