CODEWITHSUNDEEP
r
Maximilian
Maximilian

Reputation: 4229

Clear memory in loop with R

I'm running low with memory and speed as the loop proceeds. If I would place gc() in loop right after write.csv(), would that be correct and of any help?

Loop I have got:

for(i in seq_along(x) {
 ....
 ....
 write.csv(x, file=paste("C:/....",i,".csv",sep=""))
}

Upvotes: 2

Views: 6256

Answers (1)

alko989
alko989

Reputation: 7908

The garbage collector is called automatically when needed. Using gc() calls the garbage collector. I think, it makes only sense to use it if you remove objects in the loop. Then calling the garbage collector could help. Quoting from ?gc:

"[...] it can be useful to call ‘gc’ after a large object has been removed, as this may prompt R to return memory to the operating system."

Calling gc() can be time consuming. I did a little test to check that:

library(microbenchmark)
library(ggplot2)
lst <- rep(list(rnorm(10000)), 30)

res <- microbenchmark(
  for(i in seq_along(lst)) {
    write.csv(lst[[i]], file="delme.csv")
    gc()
  }, 
  for(i in seq(ll)) {
    write.csv(lst[[i]], file="delme.csv")
  })

levels(res$expr) <- c("with gc()","without gc()")
autoplot(res)

enter image description here

So it seems that calling gc() everytime is probably not a good idea. Of course it depends a lot on what you are doing in the loop.

Just a hunch: Garbage collection problems are not slowing your code down. You can probably optimize other parts of your code, e.g. using an *ply function instead of for loop can sometimes help.

Hope it helps,

alex

Upvotes: 2

Related Questions