CODEWITHSUNDEEP
rforkmulticoredata.table
Alex
Alex

Reputation: 19803

multicore and data.table in R

I am attempting to use multicore function parallel with data.table and am unable to quite come up with the right way to do this. Code:

require(multicore)
require(data.table)
dtb = data.table(a=1:10, b=1:2)
x = dtb[,parallel(a+1),by=b]

> x
   b   pid fd
1: 1 12243  3
2: 1 12243  6
3: 2 12247  4
4: 2 12247  8

I would like to call collect() on this but these are no longer parallel objects. How should one do this?

Upvotes: 7

Views: 3953

Answers (1)

eddi
eddi

Reputation: 49448

I think this is along the lines of what you want:

collect(dtb[, list(jobs = list(parallel(a+1))), by = b][, jobs])

The reason you didn't have parallel objects any more and couldn't run a collect is because you were converting them to a list, instead of storing them in a list, which is what I did above.

Upvotes: 3

Related Questions