CODEWITHSUNDEEP
rmemorymatrixstorage
A_Skelton73
A_Skelton73

Reputation: 1200

Managing a Big Matrix in R

R - I have a Square matrix of doubles 62589x62589 which when I save the matrix to an Rdata object, it is 28GB, is this an insane size or relatively normal, is there any way around this?

Upvotes: 0

Views: 778

Answers (2)

LSE
LSE

Reputation: 73

I would make sure that you've looked into R packages designed specifically for microarray data. For example, Bioconductor has packages for microarrays (http://www.bioconductor.org/help/workflows/arrays/). There are certainly others out there though. If they are designed to work with similarly large data sets then these packages may have already addressed the problem.

Upvotes: 1

krlmlr
krlmlr

Reputation: 25484

If it's a dense matrix there's not much you can do about it. The storage requirements for this matrix are roughly

6 * 6 * 10^4 * 10^4 * 8 bytes = 288 * 10^8 bytes = 28.8 Gigabytes.

8 bytes is the size for a double. You can use single values, this halves the size.

For sparse matrices, the Matrix package by Douglas Bates and Martin Maechler is your friend.

Upvotes: 3

Related Questions