CODEWITHSUNDEEP
scalastreamstack-overflowmemoization
AntoineKa
AntoineKa

Reputation: 185

Scala stream keeps intermediary objects in memory

I am trying to write a Stream in Scala and I do not get why it keeps some intermediary objects in memory (and eventually get out of memory).

Here is a simplified version of my code :

val slols = {
  def genLols (curr:Vector[Int]) :Stream[Int] = 0 #:: genLols(curr map (_ + 1))
  genLols(List.fill(1000 * 1000)(0).toVector)
}
println(slols(1000))

This seems to keep in memory the intermediary curr and I do not see why.

Here is the same code written with an iterator (much better memory consumption):

val ilols = new Iterator [Int] {
  var curr = List.fill(1000 * 1000)(0).toVector
  def hasNext = true
  def next () :Int = { curr = curr map (_ + 1) ; 0 }
}
val silols = ilols.toStream
println(silols(1000))

EDIT : I am interested in keeping the 0s in memory, my goal is to not keep the currs because they are juste useful to take a step of computation (it may not be obvious in my simplified example). And the 0s alone cannot make an out of memory error (they are not so heavy to store).

Upvotes: 0

Views: 382

Answers (1)

Aivean
Aivean

Reputation: 10882

When you assign stream to a variable you prevent it's head from being garbage collected.

In first case that means that all the references for cur Vectors for each iteration are memoized.

In second case the situation is better, as iterator stores only single instance of cur at a time. But still, all Int elements of the stream are memoized.

Try replacing val slols with def slols if you don't need memoization.

And by the way, your two examples are not equivalent in terms of logic (probably you're aware of it).

Check this article for details.

Upvotes: 2

Related Questions