CODEWITHSUNDEEP
pythondatetimepandasindexingdataframe
Lererferler
Lererferler

Reputation: 297

datetime index in python dataframe

I am writing a program to download data from a website called quandl. The issue I am having is that when the data is downloaded the index column of the dataframe that it is downloaded, is dates in the format: 2000-01-02 00:00:00. When I then use dates = df.index.values.tolist() the dates are given back as integers in the form 946684800000000000 ( this being the date from before). Does anyone know how to handle this date format and put it into a format which I can use the datetime module with???

Upvotes: 1

Views: 3804

Answers (1)

unutbu
unutbu

Reputation: 879481

If df.index is a DatetimeIndex, then use df.index.to_pydatetime() to convert the dates to an object array of datetime.datetimes. For example,

In [14]: index = pd.date_range('2000-1-1', periods=3, freq='D')

In [15]: index
Out[15]: DatetimeIndex(['2000-01-01', '2000-01-02', '2000-01-03'], dtype='datetime64[ns]', freq='D', tz=None)

In [16]: index.values.tolist()
Out[16]: [946684800000000000L, 946771200000000000L, 946857600000000000L]

In [17]: index.to_pydatetime()
Out[20]: 
array([datetime.datetime(2000, 1, 1, 0, 0),
       datetime.datetime(2000, 1, 2, 0, 0),
       datetime.datetime(2000, 1, 3, 0, 0)], dtype=object)

Note that depending on what you want to do with these dates, it may be more advantageous to work with a DatetimeIndex than an object array of datetime.datetimes.

Tip: This is the type of problem -- introspection of an unfamiliar object -- where IPython can be of great help. IPython has TAB-completion of attributes. At the IPython prompt typing

In [17]: index.

and then pressing TAB pulls up a list of all the attributes and methods of the index object that IPython has detected. (Because some objects have custom __getattr__ methods, this may not be a complete list, but it is often useful nevertheless.) Perusing the list or doing a text search for "datetime" will lead you to index.to_datetime and index.to_pydatetime. A little experimentation will then show you that index.to_pydatetime does what you need. Moreover, typing a question mark after index.to_pydatetime causes IPython to show you helpful information, including the docstring:

In [19]: index.to_pydatetime?
Type:        instancemethod
String form: <bound method DatetimeIndex.to_pydatetime of DatetimeIndex(['2000-01-01', '2000-01-02', '2000-01-03'], dtype='datetime64[ns]', freq='D', tz=None)>
File:        /home/unutbu/.virtualenvs/dev/lib/python2.7/site-packages/pandas-0.16.2+175.g5a9a9da-py2.7-linux-x86_64.egg/pandas/tseries/index.py
Definition:  index.to_pydatetime()
Docstring:
Return DatetimeIndex as object ndarray of datetime.datetime objects

Returns
-------
datetimes : ndarray

Upvotes: 3

Related Questions