CODEWITHSUNDEEP
rlistvectorsubsetlapply
Rob Marty
Rob Marty

Reputation: 398

Subset list of vectors by position in a vectorized way

I have a list of vectors and I'm trying to select (for example) the 2nd and 4th element in each vector. I can do this using lapply:

list_of_vec <- list(c(1:10), c(10:1), c(1:10), c(10:1), c(1:10))
lapply(1:length(list_of_vec), function(i) list_of_vec[[i]][c(2,4)])

[[1]]
[1] 2 4

[[2]]
[1] 9 7

[[3]]
[1] 2 4

[[4]]
[1] 9 7

[[5]]
[1] 2 4

But is there a way to do this in a vectorized way -- avoiding one of the apply functions? My problem is that my actual list_of_vec is fairly long, so lapply takes awhile.

Upvotes: 1

Views: 255

Answers (2)

Andrew
Andrew

Reputation: 5138

Solutions:

Option 1 @Athe's clever solution using do.call?:

do.call(rbind, list_of_vec)[ ,c(2,4)]

Option 2 Using lapply more efficiently:

lapply(list_of_vec, `[`, c(2, 4))

Option 3 A vectorized solution:

starts <- c(0, cumsum(lengths(list_of_vec)[-1]))
matrix(unlist(list_of_vec)[c(starts + 2, starts + 4)], ncol = 2)

Option 4 the lapply solution you wanted to improve:

lapply(1:length(list_of_vec), function(i) list_of_vec[[i]][c(2,4)])

Data:

And a few datasets I will test them on:

# The original data
list_of_vec <- list(c(1:10), c(10:1), c(1:10), c(10:1), c(1:10))

# A long list with short elements
list_of_vec2 <- rep(list_of_vec, 1e5)

# A long list with long elements
list_of_vec3 <- lapply(list_of_vec, rep, 1e3)
list_of_vec3 <- rep(list_of_vec3, 1e4)

Benchmarking:

Original list:

Unit: microseconds
 expr   min     lq     mean median    uq      max neval cld
   o1 2.276 2.8450  3.00417  2.845 3.129   10.809   100   a
   o2 2.845 3.1300  3.59018  3.414 3.414   23.325   100   a
   o3 3.698 4.1250  4.60558  4.267 4.552   20.480   100   a
   o4 5.689 5.9735 17.52222  5.974 6.258 1144.606   100   a

Longer list, short elements:

Unit: milliseconds
 expr       min        lq      mean    median        uq       max neval  cld
   o1 146.30778 146.88037 155.04077 149.89164 159.52194 184.92028    10  b  
   o2 185.40526 187.85717 192.83834 188.42749 190.32103 213.79226    10   c 
   o3  26.55091  27.27596  28.46781  27.48915  28.84041  32.19998    10 a   
   o4 407.66430 411.58054 426.87020 415.82161 437.19193 473.64265    10    d

Longer list, long elements:

Unit: milliseconds
 expr        min         lq      mean     median        uq       max neval cld
   o1 4855.59146 4978.31167 5012.0429 5025.97619 5072.9350 5095.7566    10   c
   o2   17.88133   18.60524  103.2154   21.28613  195.0087  311.4122    10 a  
   o3  855.63128  872.15011  953.8423  892.96193 1069.7526 1106.1980    10  b 
   o4   37.92927   38.87704  135.6707  124.05127  214.6217  276.5814    10 a

Summary:

Looks like the vectorized solution wins out if the list is long and the elements are short, but lapply is the clear winner for a long list with longer elements. Some of the options output a list, others a matrix. So keep in mind what you want your output to be. Good luck!!!

Upvotes: 2

Athe
Athe

Reputation: 155

If your list is composed of vectors of the same length, you could first transform it into a matrix and then get the columns you want.

matrix_of_vec <- do.call(rbind,list_of_vec)
matrix_of_vec[ ,c(2,4)]

Otherwise I'm afraid you'll have to stick to the apply family. The most efficient way to do it is using the parallel package to compute parallely (surprisingly).

corenum <- parallel::detectCores()-1
cl<-parallel::makeCluster(corenum)
parallel::clusterExport(cl,"list_of_vec"))
parallel::parSapply(cl,list_of_vec, '[', c(2,4) )

In this piece of code '[' is the name of the subsetting function and c(2,4) the argument you pass to it.

Upvotes: 1

Related Questions