CODEWITHSUNDEEP
scaladataframeapache-sparkapache-spark-sqlscala-collections
Skyflakes
Skyflakes

Reputation: 38

DATE CONVERSION SCALA

I just want to ask if it's possible to convert a string to date without changing it's format?

 .withColumn("birthdate",when(col("birthday").isNotNull,to_date(col("birthday"),"MM/dd/YYYY")).otherwise(null))

I have a string "01/01/1990" but when I use to_date(col("birthday"),"MM/dd/YYYY"), it becomes 1990-01-01

Is there any way to convert the string to date without changing it's original format or it is only the format for date datatypes?

Thank you in advanced !

Upvotes: 1

Views: 149

Answers (1)

notNull
notNull

Reputation: 31550

By default spark uses yyyy-MM-dd format for date type.

if you want to have MM/dd/YYYY format as date type then its not possible with spark (results null value).

Example:

Converting MM/dd/YYYY to date type:

scala> spark.sql("select date('01/01/1990') as dt").show()
//+----+
//|  dt|
//+----+
//|null| //result null
//+----+


scala> spark.sql("select date('01/01/1990') as dt").printSchema
//root
 //|-- dt: date (nullable = true)

Converting YYYY-MM-dd to date type:

scala> spark.sql("select date('1990-01-01') as dt").show(false)
//+----------+
//|dt        |
//+----------+
//|1990-01-01| //results data
//+----------+


scala> spark.sql("select date('1990-01-01') as dt").printSchema
//root
// |-- dt: date (nullable = true)

Upvotes: 1

Related Questions