CODEWITHSUNDEEP
scalaapache-sparkdataframe
Fugui
Fugui

Reputation: 91

Spark convert dataframe to RowMatrix

Say I have a dataframe resulted from a sequence of transformations. It looks like the following:

id matrixRow
0  [1,2,3]
1  [4,5,6]
2  [7,8,9]

each row actually corresponds to a row of a matrix. How can I convert the matrixRow column of the dataframe to RowMatrix?

Upvotes: 0

Views: 2302

Answers (1)

Fugui
Fugui

Reputation: 91

After numerous tries, here's one solution:

val rdd = df.rdd.map(
           row => Vectors.dense(row.getAs[Seq[Double]](1).toArray)//get the second column value as Seq[Double], then as Array, then cast to Vector
          )
val row = new RowMatrix(rdd)

Upvotes: 2

Related Questions