CODEWITHSUNDEEP
pythonpandasdataframeunicode
vagabond
vagabond

Reputation: 3594

python - converting unicode in list to dataframe

I am using an API to get some data. The data returned is in Unicode (not a dictionary / json object). 

# get data
data = []
for urls in api_call_list:
    data.append(requests.get(urls))

the data looks like this: 

>>> data[0].text
u'Country;Celebrity;Song Volume;CPP;Index\r\nus;Taylor Swift;33100;0.83;0.20\r\n'

>>> data[1].text
u'Country;Celebrity;Song Volume;CPP;Index\r\nus;Rihanna;28100;0.76;0.33\r\n'

I want to put this in a DataFrame with Country, Celebrity, Song, Volume, CPP and Index as column names.

First I tried to split it on \r\n like this: 

x = [i.text.split('\r\n') for i in data]

and got: 

[[u'Country;Celebrity;Song Volume;CPP;Index',
  u'us;Taylor Swift;33100;0.83;0.20',
  u''],
 [u'Country;Celebrity;Song Volume;CPP;Index',
  u'us;Rihanna;28100;0.76;0.33',
  u'']]

Not sure where to go from here . . .

Upvotes: 1

Views: 2262

Answers (1)

akuiper
akuiper

Reputation: 214937

You can use pandas.read_csv to read data as a list of data frames and then concatenate them:

# if you use python 2 change this to // from io import BytesIO and use BytesIO instead
from io import StringIO     
import pandas as pd

pd.concat([pd.read_csv(StringIO(d), sep = ";") for d in data])

enter image description here

Since your actual data is a list of responses, you may need access the text firstly:

pd.concat([pd.read_csv(StringIO(d.text), sep = ";") for d in data])

data = [u'Country;Celebrity;Song Volume;CPP;Index\r\nus;Taylor Swift;33100;0.83;0.20\r\n', 
        u'Country;Celebrity;Song Volume;CPP;Index\r\nus;Rihanna;28100;0.76;0.33\r\n']

Upvotes: 3

Related Questions