CODEWITHSUNDEEP
pythonpandas
Pawan Jain
Pawan Jain

Reputation: 825

Converting string of lists of timestamps to lists of timestamps in pandas

I parsed data from s3 which is similar to this

ID   departure
1    "[Timestamp('2021-05-25 09:00:00'), datetime.datetime(2021, 5, 25, 9, 21, 35, 769406)]"
2    "[Timestamp('2021-05-25 08:00:00'), datetime.datetime(2021, 5, 25, 11, 15), datetime.datetime(2021, 5, 25, 14, 15)]"

Is there any way to convert the departure into list

I tried this

samp['departure'] = samp['departure'].apply(lambda x: eval(x))
-> Error: eval() arg 1 must be a string, bytes or code object

and

samp['departure'] = samp['departure'].apply(lambda x: x[1:-1].split(','))
# Here datetime.datetime(2021, 5, 25, 11, 15) splited into many sub-parts

and

samp.departure = samp.departure.apply(ast.literal_eval)
error -> malformed node or string: ["Timestamp('2021-05-25 09:00:00')", ' datetime.datetime(2021', ' 5', ' 25', ' 9', ' 21', ' 35', ' 769406)']

Output should be

ID   departure
1    [Timestamp('2021-05-25 09:00:00'), datetime.datetime(2021, 5, 25, 9, 21, 35, 769406)]
2    [Timestamp('2021-05-25 08:00:00'), datetime.datetime(2021, 5, 25, 11, 15), datetime.datetime(2021, 5, 25, 14, 15)]

(I tried converters while read_csv initially but getting an error too)

Upvotes: 0

Views: 202

Answers (1)

Anurag Dabas
Anurag Dabas

Reputation: 24314

If you are trying to replace " present in your departure column:

Try via replace():

samp['departure']=samp['departure'].replace('"','',regex=True)

OR

try via strip():

samp['departure']=samp['departure'].str.strip('"')

If you are evaluating the values inside:

from pandas import Timestamp
import datetime
samp['departure']=samp['departure'].astype(str).apply(pd.eval)

OR

from pandas import Timestamp
import datetime
import ast
samp.departure = samp.departure.astype(str).apply(ast.literal_eval)

Upvotes: 1

Related Questions