mat
mat

Reputation: 2617

Parallel big matrix multiplication

I need to multiply two big matrices A and B as follow:

library(bigmemory)
library(bigalgebra)
library(biganalytics)

A <- big.matrix( replicate(100,  rnorm(10^5)) )
B <- big.matrix( replicate(10^5, rnorm(100))  )
AB <- A %*% B

How could I compute this multiplication in parallel?

The only tutorial I've come across so far is this one:

> library("doRedis")
> registerDoRedis(queue="example")
> L = foreach(j=1:2,.packages="VAM",.combine=c) %dopar%
+ {
+ key = paste("X",j,sep="")
+ ridx = ((j-1)*5 + 1):min((j*5),nrow(A))
+ X = A[ridx,] %*% B[,]
+ Y = as.big.matrix(X,backingfile=key)
+ vnew(Y, key)
+ key
+ }
> X = vam(matrix(L,nrow=2))
> sum(X[,] - A[,] %*% B[,])
[1] 0

But I'm not sure how to put it into practice. There may also be a simpler/more efficient way to achieve the same result?

Upvotes: 3

Views: 1911

Answers (1)

F. Priv&#233;
F. Priv&#233;

Reputation: 11728

Installing Microsoft R Open, I go from 3 sec to 0.1 sec!

library(bigmemory)
library(bigalgebra)

N <- 200
M <- 1e5

A <- big.matrix(N, M, init = rnorm(N * M))
B <- big.matrix(M, N, init = rnorm(N * M))
system.time(AB <- A %*% B)

Change the version of R you use in RStudio settings (need to quit RStudio afterwards)

Upvotes: 2

Related Questions