CODEWITHSUNDEEP
scala
Alex Varju
Alex Varju

Reputation: 2922

scala - lazy iterator calls next too many times?

I'm trying to build a lazy iterator that pulls from a blocking queue, and have encountered a weird problem where next() appears to be called more times than expected. Because my queue is blocking, this causes my application to get stuck in certain cases.

Some simplified sample code:

"infinite iterators" should {
  def mkIter = new Iterable[Int] {
    var i = 0
    override def iterator: Iterator[Int] = {
      new Iterator[Int] {
        override def hasNext: Boolean = true
        override def next(): Int = {
          i = i + 1
          i
        }
      }
    }
    override def toString(): String = "lazy"
  }

  "return subsets - not lazy" in {
    val x = mkIter
    x.take(2).toList must equal(List(1, 2))
    x.take(2).toList must equal(List(3, 4))
  }

  "return subsets - lazy" in {
    val x = mkIter
    x.view.take(2).toList must equal(List(1, 2))
    x.view.take(2).toList must equal(List(3, 4))
  }
}

In the example above, the lazy test fails because the second call to take(2) returns List(4, 5).

Given that I see this behaviour with both Scala 2.10 and 2.11, I suspect the error is mine, but I'm not sure what I'm missing.

Upvotes: 0

Views: 169

Answers (2)

Alex Varju
Alex Varju

Reputation: 2922

As explained by @dlwh, Scala is explicitly documented to not allow reuse of an iterator after calling take(Int). That said, a way to implement my core use case is to create a new stream each time I want to get another element out of the iterator.

Adding to my example in the original question:

"return subsets - streams" in {
  val x = mkIter
  x.toStream.take(2).toList must equal(List(1, 2))
  x.toStream.take(2).toList must equal(List(3, 4))
}

Note that toStream has the side effect of calling next() on the iterator, so this is only safe if you know you will be taking at least one item off of the stream. The advantage streams have over lazy views is that it will not call next() more than the minimum number of times needed.

Upvotes: 0

dlwh
dlwh

Reputation: 2293

take invalidates iterators. See the code example at the top of http://www.scala-lang.org/api/2.10.3/index.html#scala.collection.Iterator

Upvotes: 2

Related Questions