CODEWITHSUNDEEP
pythondatetimepandas
doctorer
doctorer

Reputation: 1782

numpy datetime64 in pandas dataframe

I have a pandas DataFrame consisting of 3 columns, with the first column an hourly timeseries of dtype np.datetime64 

import numpy as np
import pandas as pd    
timeseries = np.arange(np.datetime64('20/11/2009'), np.datetime64('19/11/2010), np.timedelta64(1, 'h'))
insolation =pd.DataFrame(columns=['datetime','GHI','DNI'])
insolation['datetime'] = timeseries

If I print(insolation) I get this output:

                   datetime  GHI  DNI
0    2009-11-20 00:00:00  NaN  NaN
1    2009-11-20 01:00:00  NaN  NaN
2    2009-11-20 02:00:00  NaN  NaN
...                  ...  ...  ...
8734 2010-11-18 22:00:00  NaN  NaN
8735 2010-11-18 23:00:00  NaN  NaN

So far, so good. But if I use this datetime64 object:

datetime=np.datetime64('2009-11-20 00:00:00')

(which does seem to exist in the DataFrame) to reference the DataFrame:

insolation[datetime][GHI] =1.0

I get an error:

KeyError: numpy.datetime64('2009-11-20T00:00:00')

Full error message is below:

Traceback (most recent call last):
  File "C:\Users\z5044992\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\indexes\base.py", line 1945, in get_loc
    return self._engine.get_loc(key)
  File "pandas\index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas\index.c:4154)
  File "pandas\index.pyx", line 159, in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)
  File "pandas\hashtable.pyx", line 675, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12368)
  File "pandas\hashtable.pyx", line 683, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12322)
KeyError: numpy.datetime64('2009-11-20T00:00:00')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/PYTHONprojects/PY_EN_Models/Scripts/create_weatherfile.py", line 109, in <module>
    main()
  File "C:/PYTHONprojects/PY_EN_Models/Scripts/create_weatherfile.py", line 87, in main
    insolation[datetime][GHI] = irradience_array[gridrow,gridcol]
  File "C:\Users\z5044992\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\frame.py", line 1997, in __getitem__
    return self._getitem_column(key)
  File "C:\Users\z5044992\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\frame.py", line 2004, in _getitem_column
    return self._get_item_cache(key)
  File "C:\Users\z5044992\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\generic.py", line 1350, in _get_item_cache
    values = self._data.get(item)
  File "C:\Users\z5044992\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py", line 3290, in get
    loc = self.items.get_loc(item)
  File "C:\Users\z5044992\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\indexes\base.py", line 1947, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas\index.pyx", line 137, in pandas.index.IndexEngine.get_loc (pandas\index.c:4154)
  File "pandas\index.pyx", line 159, in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)
  File "pandas\hashtable.pyx", line 675, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12368)
  File "pandas\hashtable.pyx", line 683, in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12322)
KeyError: numpy.datetime64('2009-11-20T00:00:00')

Process finished with exit code 1

Upvotes: 2

Views: 1716

Answers (1)

piRSquared
piRSquared

Reputation: 294516

use loc

insolation.loc[insolation.datetime.eq(datetime), 'GHI'] = 1.0

what went wrong?

insolation[datetime]

Tries to get the column defined by the variable datetime.
Tip: don't use variable names that are the same as your column names.

Upvotes: 4

Related Questions