CODEWITHSUNDEEP
scalaapache-spark
Leothorn
Leothorn

Reputation: 1345

From Array[Any] to Array[Double] in scala spark

For this code :

var p = result.select("finalFeatures").head.toSeq.toArray

The results looks like this :

p: Array[Any] = Array([3.0,6.0,-0.7876947819954485,-0.21757635218517163,0.9731844373162398,-0.6641741696340382,-0.6860072219935377,-0.2990737363481845,-0.7075863760365155,0.8188108975549018,-0.8468559840943759,-0.04349947247406488,-0.45236764452589984,1.0333959313820456,0.609756607087835,-0.7106619551471779,-0.7750330808435969,-0.08097610412658443,-0.45338437108038904,-0.2952869863393396,-0.30959772365257004,0.6988768123463287,0.17049117199049213,3.2674649019757385,-0.8333373234944124,1.8462942520757128,-0.49441222531240125,-0.44187299748074166,-0.300810826687287])

I need this to be Array[Double] How would i do this ?

Upvotes: 0

Views: 2374

Answers (2)

abiratsis
abiratsis

Reputation: 7336

Assuming you have the following data:

val df = Seq(Array("2.3", "2.0", "5")).toDF("finalFeatures")
df.show

The output for the previous command will be:

+-------------+
|finalFeatures|
+-------------+
|[2.3, 2.0, 5]|
+-------------+

df.schema will print org.apache.spark.sql.types.StructType = StructType(StructField(finalFeatures,ArrayType(StringType,true),true)) to cast the column into double array you can do:

val doubleSeq = df.select($"finalFeatures".cast("array<double>")).head.get(0).asInstanceOf[Seq[Double]]

And doubleSeq.foreach(println _) should have the next output:

2.3
2.0
5.0

Upvotes: 1

SCouto
SCouto

Reputation: 7928

You can convert an Array of Any to Double as follows:

 val pAsDouble = p.map(_.toString.toDouble)

Upvotes: 2

Related Questions