CODEWITHSUNDEEP
pythoncsvnumpytype-conversion
Jerryberry123
Jerryberry123

Reputation: 25

Converting data from a csv file into numpy array issue

In a csv file I have data representing the date, open, close, high, low, and volume for a particular stock. The data is stored in the following format:

20150601 000000;1.094990;1.095010;1.094990;1.094990;0

I am attempting to use the following code to extract the date into a numpy array so i can analyze the data using algorithms. However, when converting the date I do not get the correct date.

Can anyone identify the error that I am making?

datefunc = lambda x: mdates.date2num(datetime.strptime(x, '%y%m%d%H%M %f'))
date,high,low,open,close,volume = np.loadtxt('DAT_ASCII_EURUSD_M1_201506.csv',unpack=True, 
                              delimiter=';',
                              converters={0:datefunc})

Any help is much appreciated.

Upvotes: 2

Views: 1978

Answers (1)

Padraic Cunningham
Padraic Cunningham

Reputation: 180540

Your date format is incorrect, it needs to be year,month and day "%Y%m%d", you also cannot have a datetime object and floats in your array but using a structured array allows you to have mixed types.

If mdates returns a float using the correct format should work again providing you have a ; delimited lines:

from datetime import datetime
import numpy as np
datefunc = lambda x: mdates.date2num(datetime.strptime(x, '%Y%m%d'))

a = np.loadtxt('in.csv', delimiter=';',
                  converters={0: datefunc})

Which would output:

[  7.35750000e+05   0.00000000e+00   1.09499000e+00   1.09501000e+00
1.09499000e+00   1.09499000e+00   0.00000000e+00]

You have seven elements in your example input line so you will get an error unpacking, if that is a typo then it will be ok but if not you will need to fix it.

If you have mixed types you could use a structured array with genfromtxt :

from datetime import datetime
import numpy as np
datefunc = lambda x: datetime.strptime(x, '%Y%m%d')
a = np.genfromtxt('in.csv', delimiter=';',
              converters={0: datefunc}, dtype='object, float, float,float,float,float',
              names=["date", "high", "low", "open", "close", "volume"])

print(a["date"])
print(a["high"])
print(a["low"])
print(a["open"])
print(a["close"])
print(a["volume"])

2015-06-01 00:00:00
0.0
1.09499
1.09501
1.09499
1.09499

This presumes your input is actually delimited by ; and does not have spaces like you have in your sample line.

Upvotes: 2

Related Questions