CODEWITHSUNDEEP
apache-sparkpysparktimestamp
MetallicPriest
MetallicPriest

Reputation: 30765

Why Spark is not recognizing this time format?

I get null for the timestamp 27-04-2021 14:11 with this code. What mistake am I doing? Why is the timestamp format string DD-MM-yyyy HH:mm not correct here?

df = spark.createDataFrame([('27-04-2021 14:11',)], ['t'])
df = df.select(to_timestamp(df.t, 'DD-MM-yyyy HH:mm').alias('dt'))
display(df)

Upvotes: 0

Views: 198

Answers (1)

vladsiv
vladsiv

Reputation: 2936

D is for day of the year, and d is for day of the month.

Try this:

df = df.select(F.to_timestamp(df.t, "dd-MM-yyyy HH:mm").alias("dt"))

Upvotes: 3

Related Questions