CODEWITHSUNDEEP
rdata.table
statquant
statquant

Reputation: 14380

data.table is not handling integer64 in by statement

Using fread from data.table load integer64 correctly, though I have the impression that by statements are not handling int64 correctly. I am probably doing someting wrong here, what is it ?

library(data.table); library(bit64);
test = data.table(x=c(1,2,3),y=c('x','q','q'),ID=as.integer64(c('432706205348805058','432706205348805058','432706205348805059')))

str(test) #the display is wrong (BUT IT IS EXPECTED)
#Classes ‘data.table’ and 'data.frame':  3 obs. of  3 variables:
# $ x : num  1 2 3
# $ y : chr  "x" "q" "q"
# $ ID:Class 'integer64'  num [1:3] 9.52e-280 9.52e-280 9.52e-280
# - attr(*, ".internal.selfref")=<externalptr> 

test # Here it is displayed correctly
#   x y                 ID
#1: 1 x 432706205348805058
#2: 2 q 432706205348805058
#3: 3 q 432706205348805059

txtR) test$ID
integer64
[1] 432706205348805058 432706205348805058 432706205348805059

txtR) test[,list(count=.N),by=ID] #WRRRONG
                   ID count
1: 432706205348805058     3

Upvotes: 10

Views: 2851

Answers (1)

Matt Dowle
Matt Dowle

Reputation: 59612

Update: This is now implemented in v1.9.3 (available from R-Forge), see NEWS :

o bit64::integer64 now works in grouping and joins, #5369. Thanks to James Sams for highlighting UPCs and Clayton Stanley.
Reminder: fread() has been able to detect and read integer64 for a while.

On OP's example above:

test[, .N, by=ID]
#                    ID N
# 1: 432706205348805058 2
# 2: 432706205348805059 1

integer64 isn't yet implemented for data.table operations such as setkey or by. It was just implemented in fread only (first released to CRAN on 6 March 2013) as a first step. It could be useful as a value column for example.

I may have confused matters by filing a bug report relating to this (the one @Arun linked to). Strictly speaking, it isn't a bug but a feature request. I think of the bug list more like 'important things to resolve before the next release'.

Contributions are very welcome.

Upvotes: 8

Related Questions