CODEWITHSUNDEEP
stringscaladictionaryapache-spark
ViggieSmalls
ViggieSmalls

Reputation: 73

Spark key value pairs to string

I was wonder how I could turn key value pairs into strings for the output. Currently my code is like this:

object Task2 {
def main(args: Array[String]) {
val sc = new SparkContext(new SparkConf().setAppName("Task2"))

// read the file
val file = sc.textFile("hdfs://localhost:8020" + args(0))

val split = file
  .map(line => (line.split("\t")(1), line.split("\t")(2)))
  .reduceByKey(_ + _)

// store output on given HDFS path.
fin.saveAsTextFile("hdfs://localhost:8020" + args(1))
}
}

the split output gives me key value pairs like (x, y) but I would like them to be x y pairs separated by a tab. I've tried using map and mkString to no avail. What should I do?

Upvotes: 1

Views: 1210

Answers (1)

Justin Pihony
Justin Pihony

Reputation: 67085

This should work:

split.map(x=>x._1+"\t"+x._2).saveAsTextFile(....)

Upvotes: 2

Related Questions