CODEWITHSUNDEEP
eclipsescalaapache-spark
Azhar
Azhar

Reputation: 35

Scala code works on Spark Terminal but not in eclipse

The code below works fine when I run it on Spark Terminal but in eclipse it throws an error. What might be the reason. Please let me know if you need more information.

val IABLabels= IAB.zip(labels)
val temp1 = IABLabels.groupBy(x=>x._2).mapValues( _.map( _._1 ))

Error in Eclipse:
value mapValues is not a member of org.apache.spark.rdd.RDD[(Int, Iterable[(String, Int)])]

The code runs perfectly fine on Spark shell.

Upvotes: 0

Views: 985

Answers (1)

jarandaf
jarandaf

Reputation: 4427

You should use this import to access extra functions on RDD's of (key,value) pairs through an implicit conversion:

import org.apache.spark.SparkContext._

You can check the API docs for further details.

Upvotes: 7

Related Questions