CODEWITHSUNDEEP
rsumdata.tablerowsampling
LeGeniusII
LeGeniusII

Reputation: 960

R data.table fast sum of random samples by row

I have:

require(data.table)

dataDT <- data.table(ID = 1:4, Num_Times = c(7, 9, 10, 13))
dataDT              # the main data
   ID Num_Times
1:  1         7
2:  2         9
3:  3        10
4:  4        13


probabilityDT <- data.table(val = 1:3, prob = c(0.5, 0.3, 0.2))
probabilityDT       # the probabilty matrix
   val prob
1:   1  0.5
2:   2  0.3
3:   3  0.2

I would like to do the following:

For each row, sample and calculate the sum.

valTemp <- c()
set.seed(999)
for (i in 1:nrow(dataDT)) {

  # sample size
  num_times <- dataDT[i, Num_Times]

  # get samples
  Temp1 <- sample(x = probabilityDT[["val"]], size = num_times, replace = TRUE, prob = probabilityDT[["prob"]])

  # get sum
  Temp1 <- sum(Temp1)

  valTemp <- c(valTemp, Temp1)
}

dataDT[, sample_sum := valTemp]
dataDT
   ID Num_Times sample_sum
1:  1         7         12
2:  2         9         14
3:  3        10         20
4:  4        13         25

How to do it more efficiently? I have ~500k rows. Can this operation be vectorized at all?

Upvotes: 1

Views: 181

Answers (1)

Roland
Roland

Reputation: 132706

Check out if this is faster:

set.seed(999)
sample_all <- sample(probabilityDT[["val"]], #
                     size = sum(dataDT[["Num_Times"]]), #get all values sampled at once
                     TRUE, probabilityDT[["prob"]])

res <- data.table(sample_all, ID = rep(dataDT[["ID"]], dataDT[["Num_Times"]]))
res <- res[, .(sample_sum = sum(sample_all)), by = "ID"]

dataDT[res, sample_sum := i.sample_sum, on = "ID"]
#   ID Num_Times sample_sum
#1:  1         7         12
#2:  2         9         14
#3:  3        10         20
#4:  4        13         25

Upvotes: 2

Related Questions