CODEWITHSUNDEEP
pythoncsvnumpypython-requestsurllib2
Rob K
Rob K

Reputation: 63

Numpy loadtxt works with urllib2 response but not requests response

I am attempting to load a csv file from a url such as http://real-chart.finance.yahoo.com/table.csv?s=PXD&d=9&e=17&f=2015&g=d&a=11&b=12&c=1970&ignore=.csv into a numpy array using loadtxt. If I use urllib2.urlopen(url) this works fine but I am running into errors with requests.get(url)

Example:

file = urllib2.urlopen(url)
Date,Open,High,Low,Close,Volume,AdjClose = np.loadtxt(file, unpack=True, delimiter=',', skiprows=1,converters={0:mdates.strpdate2num('%Y-%m-%d')})
print AdjClose[:5]

Returns:

[ 140.570007  136.580002  133.240005  131.889999]

However doing the same with:

file = requests.get(url)

Results in errors such as this no matter what changes I have been attempting to make to the parameters:

ValueError: time data '33.00' does not match format '%Y-%m-%d'

Adding .text to requests.get(url) results in one long string of the entire data set along with \n characters.

Any ideas? Thanks!

Upvotes: 3

Views: 2272

Answers (1)

xnx
xnx

Reputation: 25548

With requests, I think you need to explicitly iterate over the lines of the response, otherwise loadtxt won't pick up the individual rows properly. Try:

Date,Open,High,Low,Close,Volume,AdjClose = np.loadtxt(file.iter_lines(),
            unpack=True, delimiter=',', skiprows=1,
            converters={0:mdates.strpdate2num('%Y-%m-%d')})

Upvotes: 1

Related Questions