CODEWITHSUNDEEP
scaladateapache-sparkapache-spark-sql
OUMOUSS_ELMEHDI
OUMOUSS_ELMEHDI

Reputation: 519

converting dd-MMM-yy date format in Spark

I have a Spark data frame (Scala API) which contains a column called transfer date, the dates are in string format and are in this format 24-JUL-17.

I want to convert it to date string into timestamp. How can I do it?

Upvotes: 5

Views: 7211

Answers (2)

user9518134
user9518134

Reputation:

It can also be done with to_timestamp(date_col, expr)

import org.apache.spark.sql.functions.to_timestamp

val df = date.withColumn("ts", to_timestamp($"transfer date", "dd-MMM-yy"))

Now ts column in df is of timestamp type.

Upvotes: 5

OUMOUSS_ELMEHDI
OUMOUSS_ELMEHDI

Reputation: 519

I found it :

import org.apache.spark.sql.functions.unix_timestamp

val ts = unix_timestamp($"transfer date", "dd-MMM-yy").cast("timestamp")
dfs.withColumn("ts",ts).show()

Upvotes: 4

Related Questions