CODEWITHSUNDEEP
scalaapache-sparkhashmap
Vinod Thorwat
Vinod Thorwat

Reputation: 21

Convert Resulting Rdd into HashMap

I need to convert rdd into hashMap. I am having data in RDD like below:

(FRUIT, List(Apple,Banana,Mango)) (VEGETABLE, List(Potato,Tomato))

I am having below code currently

object JsonParse {

 def main(args: Array[String]){


val sc = new SparkContext(new SparkConf().setAppName("JsonParse").setMaster("local"))

val arr = Array(("FRUIT",List("Apple","Banana","Mango")),("VEGETABLE", List("Potato","Tomato")))

val Rdd = sc.parallelize(arr)

how to proceed further??

Upvotes: 2

Views: 3123

Answers (2)

Paweł Jurczenko
Paweł Jurczenko

Reputation: 4471

rdd.collectAsMap() // Map(VEGETABLE -> List(Potato, Tomato), FRUIT -> List(Apple, Banana, Mango))

Upvotes: 4

echo
echo

Reputation: 1291

Try 

rdd.collect.toMap

to convert it to a Map. Action collect gathers the contents of the rdd locally to the master node.

Upvotes: 1

Related Questions