CODEWITHSUNDEEP
apache-sparkapache-spark-sqltimestamptimezonetimezone-offset
Georg Heiler
Georg Heiler

Reputation: 17676

Spark parse string to timestamp with timezone

I have a string like:

2018-03-21T08:15:00+01:00

and wonder how to preserve the time zone / shift from UTC when parsing it in Spark.

Seq("2018-03-21T08:15:00+01:00").toDF.select('value, to_timestamp('value, "yyy-MM-ddTHH:mm:ss")).show(false)

unfortunately only yields null. Even my format string which is omitting the shift only returns null.

Upvotes: 5

Views: 8420

Answers (1)

Alper t. Turker
Alper t. Turker

Reputation: 35219

T is not a format specifier so it should be escaped:

"yyyy-MM-dd'T'HH:mm:ss"

and timezone is denoted by X

"yyy-MM-dd'T'HH:mm:ssXXX"

Upvotes: 8

Related Questions