CODEWITHSUNDEEP
scalaapache-sparkapache-spark-sql
Markus
Markus

Reputation: 3782

How to calculate the power of 2 for the column of DataFrame

I need to calculate the power of 2 for the column p using Spark 2.2 and Scala:

But if I do it this way, I get the error, because ($"ki" / $"ni") is the column, not Double.

df.withColumn("p",(lit(1) - scala.math.pow(($"ki" / $"ni").as[Double],2))

Upvotes: 6

Views: 6944

Answers (2)

GPopat
GPopat

Reputation: 475

POW function with sample data

val someDF = Seq(
     (8,"A"),
     (12,"B"),
     (16,"C")).toDF("number","letter")

someDF.withColumn("pow_example",pow(2,$"number")).show()

Upvotes: 0

Ramesh Maharjan
Ramesh Maharjan

Reputation: 41957

you can use inbuilt pow function as 

import org.apache.spark.sql.functions._
df.withColumn("power_of_two", pow($"p", lit(2)))

Upvotes: 9

Related Questions