CODEWITHSUNDEEP
pyspark
nbv
nbv

Reputation: 23

Pyspark convert decimal to date

I see there are methods to convert string to date, but I don't see any way we can convert decimal to string. In one of the tables I am working on, date is in format 20170924.00000. Is it possible to convert this into a valid date format?

Thanks.

Upvotes: 2

Views: 1843

Answers (1)

murtihash
murtihash

Reputation: 8410

You could convert to int, then cast to string, then specify input format to to_date function to get your output.

(welcome to SO) 

df.show()

+-----------+
|date       |
+-----------+
| 20170924.0|
+-----------+ 



from pyspark.sql import functions as F
df.withColumn("date", F.to_date(F.col("date").cast("int").cast("string"), "yyyyMMdd")).show()

+----------+
|      date|
+----------+
|2017-09-24|
+----------+

Upvotes: 1

Related Questions