CODEWITHSUNDEEP
apache-spark
Pi Pi
Pi Pi

Reputation: 861

How to do some calculation between Columns in spark Dataframe?

For example, I want to plus the la and lon column and output result in another column.

+------+------------------+------------------+
|userid|               la |               lon|
+------+------------------+------------------+
|    u3|               2.0|               2.0|
|    u4|               1.0|               1.0|
|    u5|               2.0|               2.0|
|    u1|1.6666666666666667|2.6666666666666665|
|    u6|               1.0|               3.5|
|    u2|               3.0|               4.0|
+------+------------------+------------------+

Upvotes: 1

Views: 4543

Answers (1)

eliasah
eliasah

Reputation: 40360

If you just need to sum two columns together, it's pretty straightforward :

df.withColumn("x", $"la" + $"lon")

x is the name of the new column.

To elevate the column into the power of 2 :

df.withColumn("x", pow($"la" + $"lon", 2))

Upvotes: 2

Related Questions