CODEWITHSUNDEEP
scalaapache-sparkrdd
J.Done
J.Done

Reputation: 3033

Spark - filter vs List added

val rtnRdd = originRdd.filter( ~~~ ) // 1

// 2
var eventList: List[myType] = Nil
originRdd.foreach{
    if( some condition)
        eventList :+= myType( ~~ )
}
// eventList convert to RDD

Which way is proper and fast way in spark? if '1' is proper way, why shouldn't I use '2' code style?

Upvotes: 0

Views: 48

Answers (1)

Mahesh Chand
Mahesh Chand

Reputation: 3250

2nd style is not favored because functional programming leans towards more using expression over statements. In second statements, we are using statements. Statements causes side effects. Besides that, you are doing assignment programming which also causes side effects. It becomes hard to parallelize it. For more info refer

Upvotes: 1

Related Questions