CODEWITHSUNDEEP
scalaapache-spark
user6781116
user6781116

Reputation: 13

Element wise addition in spark rdd

I have an RDD of RDD[Array[Array[Double]]]. So essentially each element is a matrix. I need to do element wise addition.

so if the first element of the rdd is 

    1 2
    3 4

and second element is 

   5 6
   7 8

at the end i need to have

   6 8
   10 12

I have looked into zip but not sure if I can use it for this case.

Upvotes: 1

Views: 986

Answers (1)

Harald Gliebe
Harald Gliebe

Reputation: 7564

Yes, you can use zip, but you'll have to use it twice, once for the rows and once for the columns:

val rdd = sc.parallelize(List(Array(Array(1.0,2.0),Array(3.0,4.0)), 
                              Array(Array(5.0,6.0),Array(7.0,8.0))))

rdd.reduce((a,b) => a.zip(b).map {case (c,d) => c.zip(d).map{ case (x,y) => x+y}})

Upvotes: 1

Related Questions