CODEWITHSUNDEEP
pythonpandas
zach
zach

Reputation: 30983

list to string in df.to-csv()

I have a dataframe which includes a column that has a list. When I write the dataframe to a file then re-open it, I end up converting the list to a string. Is there a way to safely read/write dataframes that have lists as members?

df1 = DataFrame({'a':[['john quincy', 'tom jones', 'jerry rice'],['bob smith','sally ride','little wayne'],['seven','eight','nine'],['ten','eleven','twelve']],'b':[9,2,4,5], 'c': [7,3,0,9]})

df1.to_csv('temp.csv')
df2 = read_csv('temp.csv')

#note how the list (df1) has been converted to a string (df2)
df1['a'][0]
['john quincy', 'tom jones', 'jerry rice']

df2['a'][0]
"['john quincy', 'tom jones', 'jerry rice']"

Upvotes: 4

Views: 817

Answers (2)

root
root

Reputation: 80346

No need to covert the list into string in the first place, list's will be converted to string automatically. Just write the dataframe containing a list, and use ast.literal_eval on df2

                                             a  b  c
0   ['john quincy', 'tom jones', 'jerry rice']  9  7
1  ['bob smith', 'sally ride', 'little wayne']  2  3
2                   ['seven', 'eight', 'nine']  4  0
3                  ['ten', 'eleven', 'twelve']  5  9

df1.to_csv('temp.csv')
df2 = read_csv('temp.csv')

Use ast.literal_eval to get the string back to list:

import ast
fd2['a']=df2['a'].apply(lambda x: ast.literal_eval(x))
type(df2['a'][1])

Output:

list

Upvotes: 2

Jon Clements
Jon Clements

Reputation: 142106

The problem is here:

df2['a'] =df2['a'].map(f)
                   ^^^^^^

Where f = lambda x : ','.join(x)

There's no point joining it again, you want to split it to a list:

df2['a'] = df2['a'].map(lambda L: L.split(','))

Upvotes: 1

Related Questions