CODEWITHSUNDEEP
performancerdataframesubset
zx8754
zx8754

Reputation: 56169

Most efficient way of subsetting dataframes

Can anyone suggest more efficient way of subsetting dataframe without using SQL/indexing/data.table options?

I looked for similar questions, and this one suggests indexing option.

Here are ways to subset with timings.

#Dummy data
dat <- data.frame(x = runif(1000000, 1, 1000), y=runif(1000000, 1, 1000))

#Subset and time
system.time(x <- dat[dat$x > 500, ])
#   user  system elapsed 
#  0.092   0.000   0.090 
system.time(x <- dat[which(dat$x > 500), ])
#   user  system elapsed 
#  0.040   0.032   0.070 
system.time(x <- subset(dat, x > 500))
#   user  system elapsed 
#  0.108   0.004   0.109

EDIT: As Roland suggested I used microbenchmark. It seems which performs the best.

library("ggplot2")
library("microbenchmark")

#Dummy data
dat <- data.frame(x = runif(1000000, 1, 1000), y=runif(1000000, 1, 1000))

#Benchmark
res <- microbenchmark( dat[dat$x > 500, ],
                       dat[which(dat$x > 500), ],
                       subset(dat, x > 500))
#plot
autoplot.microbenchmark(res)

enter image description here

Upvotes: 4

Views: 466

Answers (1)

zx8754
zx8754

Reputation: 56169

As Roland suggested I used microbenchmark. It seems which performs the best.

library("ggplot2")
library("microbenchmark")

#Dummy data
dat <- data.frame(x = runif(1000000, 1, 1000), y=runif(1000000, 1, 1000))

#Benchmark
res <- microbenchmark( dat[dat$x > 500, ],
                       dat[which(dat$x > 500), ],
                       subset(dat, x > 500))
#plot
autoplot.microbenchmark(res)

enter image description here

Upvotes: 1

Related Questions