CODEWITHSUNDEEP
scaladataframecasting
Lilly
Lilly

Reputation: 988

cast string column to decimal in scala dataframe

I have a dataframe (scala) I am using both pyspark and scala in a notebook

#pyspark
spark.read.csv(output_path + '/dealer', header = True).createOrReplaceTempView('dealer_dl')

%scala
import org.apache.spark.sql.functions._
val df = spark.sql("select * from dealer_dl")

How to convert a string column (amount) into decimal in scala dataframe.

I tried as below.

 %scala
 df = df.withColumn("amount", $"amount".cast(DecimalType(9,2)))

But I am getting an error as below:

error: reassignment to val

I am used to pyspark and quite new to scala. I need to do by scala to proceed further. Please let me know. Thanks.

Upvotes: 0

Views: 2388

Answers (1)

Boris Azanov
Boris Azanov

Reputation: 4481

in scala you can't reasign references defined as val but val is immutable reference. if you want to use reasigning some ref you can use var but better solution is not reasign something to the same reference name and use another val. For example:

val dfWithDecimalAmount = df.withColumn("amount", $"amount".cast(DecimalType(9,2)))

Upvotes: 1

Related Questions