CODEWITHSUNDEEP
scalaapache-spark
animal
animal

Reputation: 1004

Not able to see RDD contents

i am using scala to create an RDD but when i am trying to see the contents of RDD i am getting below results 

MapPartitionsRDD[25] at map at <console>:96

I want to see the contents of RDD how can i see that ?

below is my scala code:

 object WordCount {
   def main(args: Array[String]): Unit = {
     val textfile = sc.textFile("/user/cloudera/xxx/File")
     val word = textfile.filter(x => x.length >  0).map(_.split('|'))
     println(word)
   }
}

Upvotes: 1

Views: 765

Answers (1)

Yuval Itzchakov
Yuval Itzchakov

Reputation: 149626

You need to provide an output transformation (action). e.g. use RDD.collect:

object WordCount {
   def main(args: Array[String]): Unit = {
     val textfile = sc.textFile("/user/cloudera/xxx/File")
     val word = textfile.filter(x => x.length >  0).map(_.split('|'))
     word.collect().foreach(println)
   }
}

If you have an Array[Array[T]], you'll need to flatten before using foreach:

word.collect().flatten.foreach(println)

Upvotes: 2

Related Questions