CODEWITHSUNDEEP
pythonpyarrow
thotam
thotam

Reputation: 1009

Converting string timestamp to datetime using pyarrow

Is there any possibility to convert string timestamp in pyarrow table to datetime format before writing to parquet file?

Upvotes: 3

Views: 7960

Answers (1)

taras
taras

Reputation: 6915

Depending on the timestamp format, you can make use of pyarrow.compute.strptime function. It is not well-documented yet, but you can use something like this:

import pyarrow.compute as pc
pc.strptime(table.column("Timestamp"), format='%Y-%m-%d %H:%M:%S', unit='s')

provided your data is stored in table and "Timestamp" is the name of the column with timestamp strings.

Upvotes: 5

Related Questions