miniQ
miniQ

Reputation: 721

Using tuple as a key in scala

I have not included the whole code, basically mapAgainstValue contains userId as key and list of friends as values. I want to recreate a map RDD with the following transformation in the key. What would be the reason for empty map?

var mapAgainstValue = logData.map(x=>x.split("\t")).filter(x => x.length == 2).map(x => (x(0),x(1).split(",")))
     var map:Map[String,List[String]] = Map()
            var changedMap = mapAgainstValue.map{
              line =>
                var key ="";
                for(userIds <- line._2){
                    if(line._1.toInt < userIds.toInt){
                      key =line._1.concat("-"+userIds);
                    }
                    else {
                      key = userIds.concat("-" + line._1);
                    }
                  map += (key -> line._2.toList)
                }
            }
            changedMap.collect()
            map.foreach(println)

Upvotes: 3

Views: 4902

Answers (1)

Shankar
Shankar

Reputation: 8967

Yes, you can use Tuple as a key in Map.

For example:

val userMap = Map(
    (1, 25) -> "shankar",
    (2, 35) -> "ramesh")

Then you can try print the output using foreach

val userMapRDD = sparkContext.parallelize(userMap.toSeq, 2)
  mapRDD.foreach(element => {
    println(element) 
  })

If you want to transform the mapRDD to something else. following code returns only age and name as tuple.

  val mappedRDD = userMapRDD.map {
    case ((empId: Int, age: Int), name: String) => {
      (age, name)
    }
  }

Upvotes: 5

Related Questions