CODEWITHSUNDEEP
apache-sparkpyspark
Partha Sarkar
Partha Sarkar

Reputation: 367

pyspark convert dataframe column from timestamp to string of "YYYY-MM-DD" format

In pyspark is there a way to convert a dataframe column of timestamp datatype to a string of format 'YYYY-MM-DD' format?

Upvotes: 20

Views: 46568

Answers (4)

Arun Mohan
Arun Mohan

Reputation: 347

one other option to try out will be

from pyspark.sql import functions as F

df = df.withColumn('new_time_column', F.to_timestamp(df['Time_column'], 'yyyy-MM-dd'))

Upvotes: 0

Ramesh Maharjan
Ramesh Maharjan

Reputation: 41987

If you have a column with schema as 

root
 |-- date: timestamp (nullable = true)

Then you can use from_unixtime function to convert the timestamp to string after converting the timestamp to bigInt using unix_timestamp function as 

from pyspark.sql import functions as f
df.withColumn("date", f.from_unixtime(f.unix_timestamp(df.date), "yyyy-MM-dd"))

and you should have 

root
 |-- date: string (nullable = true)

Upvotes: 12

Frank
Frank

Reputation: 542

from pyspark.sql.functions  import date_format

df.withColumn("DateOnly", date_format('DateTime', "yyyy-MM-dd")).show()

Upvotes: 0

koiralo
koiralo

Reputation: 23119

You can use date_format function as below 

from pyspark.sql.functions import date_format

df.withColumn("dateColumn",  date_format(col("vacationdate"), "yyyy-MM-dd"))

Hope this helps!

Upvotes: 30

Related Questions