CODEWITHSUNDEEP
apache-sparkfiltersplit
Learner
Learner

Reputation: 33

How to filter after split() in rdd spark scala?

1,John,NY
2,Bill,FL
3,Harry,TX

I have a textfile with above data.

val rdd = sc.textFile("/path").map(x=>(x.split(",")(0).toInt,x.split(",")(1),x.split(",")(2)))

After this how can i filter data where name is Bill or number is > 2 ? Or is there another way to do it without split function?

Upvotes: 0

Views: 312

Answers (1)

s.polam
s.polam

Reputation: 10372

Use filter function.

df
.map(x=>(x.split(",")(0).toInt,x.split(",")(1),x.split(",")(2)))
.filter(row => row._2 == "Bill" || row._1 > 2)

Upvotes: 1

Related Questions