Trogdor
Trogdor

Reputation: 3

How do I get date and time from two columns of a numpy array into a datetime format that can be used by matplotlib?

I have a .csv file of data where the first column is date in the mm/dd/yy format, and the second column is time in the hh:mm:ss format each column after that is a series of temperature readings at a different location.

I am trying to plot a time series of temperature versus time, but the data spans multiple days, so I need both date and time on the x-axis, but I can't seem to get an array of dates and an array of times to combine into an array of datetimes, which datetime.strptime seems to need before if can turn the datetime into something matplotlib can use.

Currently I have tried to split everything into arrays of intergers, but I am not sure where I would go from there. (And I don't know how to paste my code without weird formatting, sorry.

Note: I have only started learning python just over two weeks ago, and ~50% of everything I've learned to do so far has been thanks to this site, so thank you already. But I couldnt seem to find a workable solution to my situation, so I am asking this question.

Edit: Thanks to Ajean for code posting help, I was doing it backwards. Here is the code where I was playing around with breaking it into integers.

#datetime test
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import datetime as dt

m8=np.genfromtxt('datetime test.csv', delimiter=",", skip_header=1, skip_footer=0, usecols=10)
#uses data from meter 8


d=np.genfromtxt('datetime test.csv', delimiter=",", skip_header=1, skip_footer=0, usecols=0, dtype=None)
date=np.genfromtxt(d, delimiter='/', dtype=[('month', int), ('day', int), ('year', int)])
#parse date colum into array of months days and years
month=date['month']
day=date['day']
year=date['year']
#create individual arrays of integers


t=np.genfromtxt('datetime test.csv', delimiter=",", skip_header=1, skip_footer=0, usecols=1, dtype=None)
time=np.genfromtxt(t, delimiter=':', dtype=[('hr', int), ('min', int), ('sec', int)])
#parse time column into array of hours minutes and seconds
hour=time['hr']
minute=time['min']
second=time['sec']
#create seperate arrays of ints

Here is the main code, there is some stuff that didn't work commented out that I hadn't deleted yet, and I think some code not commented out that is doing anything useful at the moment:

import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import datetime as dt

#Fiber data

tf10=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=",", skip_header=1, skip_footer=0, usecols=91)
#uses data from meter 10, 79=0m

tf100=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=",", skip_header=1, skip_footer=0, usecols=181)
#uses data from meter 10, 79=0m

tf300=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=",", skip_header=1, skip_footer=0, usecols=381)
#uses data from meter 10, 79=0m

t=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=',', skip_header=1, skip_footer=0, usecols=1, dtype=None)
tseg=np.genfromtxt(t, delimiter=':', dtype=[('hr', int), ('min', int), ('sec', int)])
#uses time codes
time=tseg['hr']*3600+tseg['min']*60+tseg['sec']
#turn time ints into # of seconds

d=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=',', skip_header=1, skip_footer=0, usecols=0, dtype=None)
dseg=np.genfromtxt(d, delimiter='/', dtype=[('month', int), ('day', int), ('year', int)])
#uses time codes
#time=dt.datetime(dseg['year'], dseg['month'], dseg['day'], tseg['hr'], tseg['min'], tseg['sec'])


#PLOT

fig = plt.figure()

p1 = fig.add_subplot(311)
p1.plot(time, tf10)
#p1.plot(time, tf100)
p1.set_title('10 Meter')
#plot and title for 10m

p2 = fig.add_subplot(312, sharey=p1)#use y axis of plot 1
p2.plot(time, tf100)
p2.set_title('100 Meter')
#plot and title for 100m
p2.set_ylabel('Temperature ($^\circ$C)')

p3 = fig.add_subplot(313, sharey=p1)#use y axis of plot 1
p3.plot(time, tf300)
p3.set_title('300 Meter')
#plot and title for 300m
p3.set_xlabel('Time of Day (hr)')

plt.show()
#Diplay plot

This code currently gives three plots in one figure with the time series cycling through "one" day over and over, like so:

enter image description here

Here is a small sample of the .csv, but there are really ~400 columns and ~4000 rows (and let me know if there is a better way to format the .csv data)

6/27/15 23:57:27    33.74   21.73   22.26   22.44   22.1

6/27/15 23:57:53    33.74   21.41   22.22   21.99   21.85

6/27/15 23:58:19    33.74   21.19   21.68   21.91   21.44

6/27/15 23:58:45    33.74   20.93   21.48   21.35   21.32

6/27/15 23:59:11    33.73   20.44   21.1    21.3    21.14

6/27/15 23:59:37    33.71   20.45   21.08   21.5    20.81

6/28/15 0:00:03 33.68   20.56   20.86   21.31   20.72

6/28/15 0:00:25 33.68   20.38   20.93   21.3    20.62

6/28/15 0:00:42 33.68   20.47   20.92   21  20.59

6/28/15 0:00:59 33.68   20.32   20.84   20.83   20.79

EDIT 2:

Figured out part of it, but still don't have whole solution, here is how I got the the strings combined in one array, with a space separating them:

import numpy as np
import datetime as dt

#Fiber data

t=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=',', skip_header=1, skip_footer=0, usecols=1, dtype=None)
#time array
d=np.genfromtxt('2015-06-30-2015-07-01-fiberdata.CSV', delimiter=',', skip_header=1, skip_footer=0, usecols=0, dtype=None)
#date array

space=' '
dspace=[x+space for x in d]
#allows space to be added to end of each date string for later date-time separation

dtcode=np.core.defchararray.add(dspace, t)
#outputs date followed by time, space between
#format: 'mm/dd/yy HH:MM:SS'
print dtcode

dtfix=dt.datetime.strptime(dtcode, "%m/%d/%y %H:%M:%S")
print dtfix

dtcode outputs properly is the format specified in the comment, but the last part won't work on arrays apparently,

TypeError: must be string, not numpy.ndarray So I still don't have an answer for my overall question.

Upvotes: 0

Views: 3203

Answers (1)

caleb c.
caleb c.

Reputation: 327

Change your code to this:

dtfix=[dt.datetime.strptime(x, "%m/%d/%y %H:%M:%S") for x in dtcode]

That calls strptime for each dtcode.

Upvotes: 2

Related Questions