CODEWITHSUNDEEP
pythondataframepython-requests
jake wong
jake wong

Reputation: 5228

python requests.text to pandas dataframe

I have the below code, where I am trying to get the data from https://www.quandl.com/data/TSE/documentation/metadata. (Trying to get the Download detailed data)

for page_number in range(1, 5):
    link = r'https://www.quandl.com/api/v3/datasets.csv?database_code=TSE&per_page=100&sort_by=id&page=' + str(page_number)
    r = requests.get(link, stream=True).text
    print(r)
    # How to put the results in a dataframe?

However, I have trouble putting the results in a dataframe / saving it in a SQLite database. How should I be doing this?

Upvotes: 2

Views: 3777

Answers (1)

user6022341
user6022341

Reputation:

You can use Pandas to read this data directly:

import pandas as pd

url = ("https://www.quandl.com/api/v3/datasets.csv?"
       "database_code=TSE&per_page=100&sort_by=id&page={0}")
[pd.read_csv(url.format(page_number)) for page_number in range(1, 5)]

To read from response you can use StringIO:

from io import StringIO

pd.read_csv(StringIO(r.text))

Upvotes: 7

Related Questions