CODEWITHSUNDEEP
pythonmongodbpandasdatetime-format
G.W.
G.W.

Reputation: 43

How to Insert datetime index of pandas DataFrame into mongodb as a field with isoDate type

I have a dataframe with type pandas.core.frame.DataFrame like: 

              open  high  close   low    volume     amount
 date                                                    
 2015-11-27  8.10  8.16   7.35  7.34  37877648  294274400
 2015-11-26  8.03  8.44   8.16  8.00  45933600  378897088
 2015-11-25  7.95  8.03   7.99  7.89  21255886  169172176
 2015-11-24  7.95  8.18   8.04  7.85  24808112  199120256
 2015-11-23  8.24  8.29   7.95  7.93  29176848  236019264

df.index

DatetimeIndex(['2015-11-27', '2015-11-26', '2015-11-25', '2015-11-24',
           '2015-11-23', '2015-11-20', '2015-11-19', '2015-11-18',
           '2015-11-17', '2015-11-16',
           ...
           '2014-12-12', '2014-12-11', '2014-12-10', '2014-12-09',
           '2014-12-08', '2014-12-05', '2014-12-04', '2014-12-03',
           '2014-12-02', '2014-12-01'],
          dtype='datetime64[ns]', name=u'date', length=193, freq=None)

How to insert df into mongodb with a date.index field in isoDate format but not datatime64[ns]/timestamps? .

Upvotes: 1

Views: 3025

Answers (1)

giser_yugang
giser_yugang

Reputation: 6176

I've been bothered for a long time, but I finally found a way to do it. It depends on the way we insert mongod. I used to insert mongod in this way:

records = json.loads(DataFrame.T.to_json()).values()
collection.insert_many(records)

We need to understand that this approach will convert DataFrame time into datatime64[ns]/timestamps.

print records[0]
{u'dtime': 1407076151000L,u'olddtime': u'2014/8/3 14:29:11'}

We can see that it saves time as numberlong and DateTime as string.So we need change records:

for row in records:
    row['olddtime'] = datetime.datetime.strptime(row['olddtime'], "%Y/%m/%d %H:%M:%S")

Then we print it:

{u'olddtime': datetime.datetime(2014, 8, 3, 14, 29, 11), u'dtime': 1407076151000L}

Finally we do this:

collection.insert_many(records)

That's OK!

Upvotes: 3

Related Questions