CODEWITHSUNDEEP
dataframepysparkdatabricks
vin
vin

Reputation: 195

pyspark convert column in dataframe to datetime with no colons for time

I have a column in a dataframe that looks like "20221026 220032". I need to convert this to a datetime column but I don't have the colon separators for the time portion. I've attempted the following:

df_meta = df_meta.withColumn("filedatetime",F.to_timestamp(F.col("filedatetime_str"),"YYYYddmm HHmmss"))

But I get an error that it is an unrecognizable format. The reason my column comes this way is because is was parsed from a filename. Is there a way to convert this to a datetime without having to insert the colons?

Upvotes: 1

Views: 169

Answers (1)

过过招
过过招

Reputation: 4234

Try:

df_meta = df_meta.withColumn("filedatetime",F.to_timestamp(F.col("filedatetime_str"),"yMMdd HHmmss"))

spark date and time format can refer to Datetime Patterns

Upvotes: 1

Related Questions