CODEWITHSUNDEEP
scalaapache-sparkspark-streaming
Klue
Klue

Reputation: 1357

How to use math.sqrt for DStream[(Double,Double)]?

For the streaming data DStream[(Double, Double)], how do I estimate the root mean squared error? See my code below. The line math.sqrt(summse) is where I have a problem (the code does not compile):

  def calculateRMSE(output: DStream[(Double, Double)], n: DStream[Long]): Double = {
        val summse = output.foreachRDD { rdd =>
          rdd.map {
              case pair: (Double, Double) =>
                val err = math.abs(pair._1 - pair._2);
                err*err
          }.reduce(_ + _)
        }
        math.sqrt(summse)
  }

UPDATE: The code doesn't compile: Cannot resolve reference sqrt with such signature. Expected: Double, Actual: Unit

Upvotes: 0

Views: 292

Answers (1)

thoredge
thoredge

Reputation: 12601

The method foreachRDD(...) returns unit so that is expected. According to the docs the result is written back to the this (output) DStream. I guess it's that you'll have to apply sqrt to.

Upvotes: 1

Related Questions