CODEWITHSUNDEEP
imagermemoryconcatenation
Róisín Grannell
Róisín Grannell

Reputation: 1978

Concatenating images in R

I'm trying to plot a large matrix (8,000 x 8,000) as a raster image in R. Unfortunately this requires too much memory for my 32-bit OS, so I'm plotting the data as two (4000 x 8000) images before recombining them.

I've looked at a lot of packages, and I've found no suitable function. I know images are imported as S4 objects with the colours stored in an array, implying that there should be a way to combine them, but I can't figure it out. Does anyone know how to do this in R? Thanks

Edit:

The data is stored in 8000 csv files, with file1 corresponding to the 1st row of the matrix, file2 to the second...

example code

# get the name of each matrix-row file
# each file is a vector of length 8000, each filei corresponding to matrix row i 

a <- list.files()

for(i in 1:4000){

     # read the data into R, and combine it with the other rows

     matrixRow <- read.table(a[i])
     matrixToPlot <- rbind(matrixToPlot, matrixRow)   

 }


 png("test", 4000, 4000)
     rasterImage(as.raster(matrixToPlot))
 graphics.off()
 ## identical code for matrix-row 4001, 4002, ...8000

Upvotes: 1

Views: 660

Answers (2)

tim riffe
tim riffe

Reputation: 5691

I tried the following while eyeing the memory on my system monitor, and it stays below 3 Gb the whole time:

    A <- matrix(rnorm(8000*8000),ncol = 8000)
    print(object.size(A),units = "Mb") # 488.3 Mb
    gc() # remove wasted mem (something I don't fully understand)

    # open a plot of appropriate dimensions- scale to your case
    plot(NULL, type = "n", xlim = c(0, 8001), ylim = c(0, 8000))
    # add chunks of the image a bit at a time, removing waste memory at each chunk
    for (m in 1:4){
      for (n in 1:4){
        image(x = (m * 2000 - 2000):(m * 2000 - 1) + .5, # pixels are centered..
              y = (n * 2000 - 2000):(n * 2000 - 1) + .5, 
              t(A[(m * 2000 - 1999):(m * 2000), (n * 2000 - 1999):(n * 2000)]), 
              useRaster = TRUE, # this is key
              add = TRUE)     
        gc() # my naive guess is that this keeps wasted mem from building up
      }
    }

this gets the thing plotted in window about as economically as I can think of. You could do the same thing by only having parts of A at a time in memory.

Upvotes: 1

mdsumner
mdsumner

Reputation: 29477

Try scaling down the matrix, do you really need that detail? Doing incremental rbind like that will kill performance and also that plot won't work, rasterImage needs an existing plot set up. Try asking with your real goal in mind, this is not a good way to proceed.

Upvotes: 1

Related Questions