CODEWITHSUNDEEP
scalaapache-sparkplayframework
Mozzer
Mozzer

Reputation: 91

How can i convert a Seq in a Spark Rdd

I'm using Spark Scala and Play Framework I have a seq like this 

//a sequence of Book objects
val books:[Seq[Book]]

that i fill with the format method from a json file: 

implicit val bookFormat: Format[Libri] = {
   ((JsPath \ "City").format[String] and
    (JsPath \ "GEN").format[Int] and
    (JsPath \ "SER").format[Int]    
    ) (Libri.apply , unlift(Libri.unapply)) }

val books = Json.parse(JsonString).as[Seq[Libri]]

How can i convert this seq in a Spark RDD. (I want to use this rdd for make some query...so i need the "registerTempTable" and "rdd.sqlContext.sql"

Upvotes: 3

Views: 8053

Answers (1)

Aravindh S
Aravindh S

Reputation: 1245

You can use sparkContext.parallelize(books). parallelize takes a collection and splits it into RDD. You can pass an additional parameter to define the number of partitions into which this seq will be split.

Upvotes: 3

Related Questions