CODEWITHSUNDEEP
rmatrixdataframe
Alex
Alex

Reputation: 15718

Why does which work faster on a data frame column compared to a matrix column?

I have the following data:

height = 1:10000000
length = -(1:10000000)
body_dim = data.frame(height,length)
body_dim_mat = as.matrix(body_dim)

Why does which() work faster for the data frame compared to the matrix?

> microbenchmark(body_dim[which(body_dim$height==50000),"length"])
Unit: milliseconds
                                                expr      min       lq   median       uq      max neval
 body_dim[which(body_dim$height == 50000), "length"] 124.4586 125.1625 125.9281 127.9496 284.9824   100

> microbenchmark(body_dim_mat[which(body_dim_mat[,1] == 50000),2])
Unit: milliseconds
                                               expr      min       lq   median      uq     max neval
 body_dim_mat[which(body_dim_mat[, 1] == 50000), 2] 251.1282 252.4457 389.7251 400.313 1004.25   100

Upvotes: 3

Views: 75

Answers (2)

redmode
redmode

Reputation: 4941

It seems that problem is before which(), subsetting of data.frame column is simply faster if compared to subsetting of whole matrix:

microbenchmark(body_dim$height==50000)
# Unit: milliseconds
#                      expr      min       lq   median       uq      max neval
#  body_dim$height == 50000 138.2619 148.5132 170.1895 170.8909 249.4592   100

microbenchmark(body_dim_mat[,1]==50000)
# Unit: milliseconds
#                       expr     min       lq   median       uq      max neval
# body_dim_mat[, 1] == 50000 299.599 308.6066 310.9036 354.4641 432.7833   100

By the way, this case is where data.table can shine:

require(data.table)
dt <- data.table(body_dim, key="height")
microbenchmark(dt[J(50000)]$length, unit="ms")
# Unit: milliseconds
#                 expr     min      lq   median       uq      max neval
#  dt[J(50000)]$length 0.96637 0.97908 0.989772 1.025257 2.588402   100

Upvotes: 1

Roland
Roland

Reputation: 132864

A data.frame is a list and a column is a simple vector and very easy to extract from the list. A matrix is a vector with dimension attributes. Which values belong to one column has to be calculated from the dimensions. This effects subsetting, which you include in your benchmarks:

library(microbenchmark)

set.seed(42)
m <- matrix(rnorm(1e5), ncol=10)
DF <- as.data.frame(m)

microbenchmark(m[,1], DF[,1], DF$V1)
#Unit: microseconds
#   expr    min     lq median      uq      max neval
# m[, 1] 80.997 82.536 84.230 87.1560 1147.795   100
#DF[, 1] 15.399 16.939 20.789 22.6365  100.090   100
#  DF$V1  1.849  2.772  3.389  4.3130   90.235   100

However, the take-home message is not that you should always use a data.frame. Because if you do subsetting, where the result is not a vector:

microbenchmark(m[1:10, 1:10], DF[1:10, 1:10])
# Unit: microseconds
#           expr     min       lq   median      uq      max neval
#  m[1:10, 1:10]   1.233   1.8490   3.2345   3.697   11.087   100
# DF[1:10, 1:10] 211.267 219.7355 228.2050 252.226 1265.131   100

Upvotes: 5

Related Questions