CODEWITHSUNDEEP
scalafunctional-programmingstreamside-effects
SkyWalker
SkyWalker

Reputation: 14309

How can this Stream iterate-takeWhile code be rewritten without side effects?

I wrote a tiny website Google ranking checker in Scala 2.12.x using page scraping that finds the rank of a website given a search term. I wanted to build it using Scala's Stream and this is a control-structure mock of the code. However, I can't find a way to rewrite it without side effects, in other words, without using any var.

def main(args: Array[String]): Unit = {
  val target = 22 // normally this would be the website domain name
  val inf = 100   // we don't care for ranks above this value
  var result: Option[Int] = None // <============= Side effects! how to rewrite it?
  Stream.iterate(0)(_ + 10).takeWhile { i =>
    // assume I'm page-scraping Google with 10 results per page
    // and need to find the rank or position where the target
    // website appears
    for (j <- i until (i + 10)) {
      // check whether the website was found
      if (j == target) {
        result = Some(j)         // <============= Side effects! how to rewrite it?
      }
    }
    result.isEmpty && i < inf
  }.toList
  println(result.getOrElse(inf))
}

Basically I'd like the Stream statement to return me the result directly which is the position or rank where the target website appears. I can't iterate one by one because the code gets each page of 10 results at a time, page-scrapes them and searches for the target website within each group of 10 results.

Upvotes: 1

Views: 249

Answers (1)

Krzysztof Atłasik
Krzysztof Atłasik

Reputation: 22595

You could just split your pipeline into map and dropWhile (replaced takeWhile):

val target = 22 // normally this would be the website domain name
val inf = 100   // we don't care for ranks above this value

val result = Stream.iterate(0)(_ + 10).map { i => 
  //or maybe just use find?
   val r = Stream.range(i-10, i).dropWhile(_ != target).headOption 
  (r,i) //we pass result with index for dropWhile
}.dropWhile{
  case (r, i) => r.isEmpty && i < inf //drop while predicate is false
}.map(_._1) //take only result
  .head //this will throw an exception if nothing is found, maybe use headOption?

You should be also aware, that I just get rid of assigning a mutable variable, but your code would still have side-effects because you're doing network calls.

You should consider using Future or some kind of IO monad to handle these calls.

Upvotes: 3

Related Questions