How can I overcome "datetime.datetime not JSON serializable"?

Question

I have a basic dict as follows:

sample = {}
sample['title'] = "String"
sample['somedate'] = somedatetimehere

When I try to do jsonify(sample) I get:

TypeError: datetime.datetime(2012, 8, 8, 21, 46, 24, 862000) is not JSON serializable

What can I do such that my dictionary sample can overcome the error above?

Note: Though it may not be relevant, the dictionaries are generated from the retrieval of records out of mongodb where when I print out str(sample['somedate']), the output is 2012-08-08 21:46:24.862000.

Answer 1

My solution is simple, it uses jsonable_encoder from FastAPI to turn any object to something that can be encoded in JSON. It has a source code in the link so you don't have to install FastAPI.

json.dumps(jsonable_encoder(obj))

Answer 2

I often use IntelliJ Evaluate Expression tool during debugging to copy over certain objects to analyze them. The problem with that approach is that if dictionary that you try to dump has a date it's brakes and the other problem that you can't define a converter using that tool, so I come up with this one liner:

json.dumps(dict_to_dump, default=lambda o: o.__str__() if isinstance(o, datetime) else None)

Answer 3

You should apply the .strftime() method on a datetime.datetime object to make it a string.

Here's an example:

from datetime import datetime

time_dict = {'time': datetime.now().strftime('%Y-%m-%dT%H:%M:%S')}
sample_dict = {'a': 1, 'b': 2}
sample_dict.update(time_dict)
sample_dict

Output:

Out[0]: {'a': 1, 'b': 2, 'time': '2017-10-31T15:16:30'}

---

[UPDATE]:

In Python 3.6 or later, you can simply use the .isoformat() method:

from datetime import datetime

datetime.now().isoformat()

Answer 4

The json.dumps method can accept an optional parameter called default which is expected to be a function. Every time JSON tries to convert a value it does not know how to convert, it will call the function we passed to it. The function will receive the object in question, and it is expected to return the JSON representation of the object.

def myconverter(o):
  if isinstance(o, datetime.datetime):
    return o.__str__()

print(json.dumps(d, default = myconverter))

Answer 5

Use:

def j_serial(o):     # Self-contained
    from datetime import datetime, date
    return str(o).split('.')[0] if isinstance(o, (datetime, date)) else None

Usage of the above utility:

import datetime
serial_d = j_serial(datetime.datetime.now())
if serial_d:
    print(serial_d)  # Output: 2018-02-28 02:23:15

Answer 6

Another approach is to adopt a concept from FEEL (the Friendly Enough Expression Language) defined in DMN (Decision Model Notation) - namely @strings.

Any string starting with @" and ending with " is decoded separately with FEEL decoding. Of course the sender and the receiver have to agree to this convention, but ... the code below lets you encode lots of other things as well as dates, times, date/times, timedeltas.

You can encode year/month durations and ranges (so long as you except a 4 element tuple of chr, expr, expr, chr as being a good representation of a range - where the two chrs are open/close brackets). So, @"P4Y2M" is a duration of 4 years and 2 months. @"P2DT5H" is a timedelta of 2 days and 4 hours, @"(2021-01-02 .. 2021-12-31)" is a year range.

The following code can be used to serialize and de-serialize @strings.

import datetime
import pySFeel

parser = pySFeel.SFeelParser()


def convertAtString(thisString):
    # Convert an @string
    (status, newValue) = parser.sFeelParse(thisString[2:-1])
    if 'errors' in status:
        return thisString
    else:
        return newValue


def convertIn(newValue):
    if isinstance(newValue, dict):
        for key in newValue:
            if isinstance(newValue[key], int):
                newValue[key] = float(newValue[key])
            elif isinstance(newValue[key], str) and (newValue[key][0:2] == '@"') and (newValue[key][-1] == '"'):
                newValue[key] = convertAtString(newValue[key])
            elif isinstance(newValue[key], dict) or isinstance(newValue[key], list):
                newValue[key] = convertIn(newValue[key])
    elif isinstance(newValue, list):
        for i in range(len(newValue)):
            if isinstance(newValue[i], int):
                newValue[i] = float(newValue[i])
            elif isinstance(newValue[i], str) and (newValue[i][0:2] == '@"') and (newValue[i][-1] == '"'):
                newValue[i] = convertAtString(newValue[i])
            elif isinstance(newValue[i], dict) or isinstance(newValue[i], list):
                newValue[i] = convertIn(newValue[i])
    elif isinstance(newValue, str) and (newValue[0:2] == '@"') and (newValue[-1] == '"'):
        newValue = convertAtString(newValue)
    return newValue


  def convertOut(thisValue):
      if isinstance(thisValue, datetime.date):
          return '@"' + thisValue.isoformat() + '"'
      elif isinstance(thisValue, datetime.datetime):
          return '@"' + thisValue.isoformat(sep='T') + '"'
      elif isinstance(thisValue, datetime.time):
          return '@"' + thisValue.isoformat() + '"'
      elif isinstance(thisValue, datetime.timedelta):
          sign = ''
          duration = thisValue.total_seconds()
          if duration < 0:
              duration = -duration
              sign = '-'
          secs = duration % 60
          duration = int(duration / 60)
          mins = duration % 60
          duration = int(duration / 60)
          hours = duration % 24
          days = int(duration / 24)
          return '@"%sP%dDT%dH%dM%fS"' % (sign, days, hours, mins, secs)
      elif isinstance(thisValue, bool):
          return thisValue:
      elif thisValue is None:
          return thisValue:
      elif isinstance(thisValue, int):
          sign = ''
          if thisValue < 0:
              thisValue = -thisValue
              sign = '-'
          years = int(thisValue / 12)
          months = (thisValue % 12)
          return '@"%sP%dY%dM"' % (sign, years, months)
      elif isinstance(thisValue, tuple) and (len(thisValue) == 4):
          (lowEnd, lowVal, highVal, highEnd) = thisValue
          return '@"' + lowEnd + str(lowVal) + ' .. ' + str(highVal) + highEnd
      elif thisValue is None:
          return 'null'
      elif isinstance(thisValue, dict):
          for item in thisValue:
              thisValue[item] = convertOut(thisValue[item])
          return thisValue
      elif isinstance(thisValue, list):
          for i in range(len(thisValue)):
              thisValue[i] = convertOut(thisValue[i])
          return thisValue
      else:
          return thisValue

Answer 7

If you are working with Django models you can directly pass encoder=DjangoJSONEncoder to the field constructor. It will work like a charm.

from django.core.serializers.json import DjangoJSONEncoder
from django.db import models
from django.utils.timezone import now


class Activity(models.Model):
    diff = models.JSONField(null=True, blank=True, encoder=DjangoJSONEncoder)


diff = {
    "a": 1,
    "b": "BB",
    "c": now()
}

Activity.objects.create(diff=diff)

Answer 8

As per the jjmontes' answer, I have used the following approach. For Flask and flask-restful users

# Get JSON string
jsonStr = json.dumps(my_dictionary, indent=1, sort_keys=True, default=str)
# Then convert the JSON string to a JSON object
return json.loads(jsonStr)

Answer 9

I faced this issue today, I found something called pickle. It's a built-in library for serializing Python objects and also load it from a pickle file.

The only difference I found between pickle and json is pickle file is a binary file, whereas json is a usual text file.

And it doesn't cause any issues with datetime objects.

Answer 10

If you are using Python 3.7, then the best solution is using datetime.isoformat() and datetime.fromisoformat(); they work with both naive and aware datetime objects:

#!/usr/bin/env python3.7

from datetime import datetime
from datetime import timezone
from datetime import timedelta
import json

def default(obj):
    if isinstance(obj, datetime):
        return { '_isoformat': obj.isoformat() }
    raise TypeError('...')

def object_hook(obj):
    _isoformat = obj.get('_isoformat')
    if _isoformat is not None:
        return datetime.fromisoformat(_isoformat)
    return obj

if __name__ == '__main__':
    #d = { 'now': datetime(2000, 1, 1) }
    d = { 'now': datetime(2000, 1, 1, tzinfo=timezone(timedelta(hours=-8))) }
    s = json.dumps(d, default=default)
    print(s)
    print(d == json.loads(s, object_hook=object_hook))

output:

{"now": {"_isoformat": "2000-01-01T00:00:00-08:00"}}
True

If you are using Python 3.6 or below, and you only care about the time value (not the timezone), then you can use datetime.timestamp() and datetime.fromtimestamp() instead;

If you are using Python 3.6 or below, and you do care about the timezone, then you can get it via datetime.tzinfo, but you have to serialize this field by yourself; the easiest way to do this is to add another field _tzinfo in the serialized object;

Finally, beware of precisions in all these examples;

Answer 11

Generally there are several ways to serialize datetimes, like:

1. ISO 8601 string, short and can include timezone info, e.g., jgbarah's answer
2. Timestamp (timezone data is lost), e.g. JayTaylor's answer
3. Dictionary of properties (including timezone).

If you're okay with the last way, the json_tricks package handles dates, times and datetimes including timezones.

from datetime import datetime
from json_tricks import dumps
foo = {'title': 'String', 'datetime': datetime(2012, 8, 8, 21, 46, 24, 862000)}
dumps(foo)

which gives:

{"title": "String", "datetime": {"__datetime__": null, "year": 2012, "month": 8, "day": 8, "hour": 21, "minute": 46, "second": 24, "microsecond": 862000}}

So all you need to do is

`pip install json_tricks`

and then import from json_tricks instead of json.

The advantage of not storing it as a single string, int or float comes when decoding: if you encounter just a string or especially int or float, you need to know something about the data to know if it's a datetime. As a dict, you can store metadata so it can be decoded automatically, which is what json_tricks does for you. It's also easily editable for humans.

Disclaimer: it's made by me. Because I had the same problem.

Answer 12

This question repeats time and time again—a simple way to patch the json module such that serialization would support datetime.

import json
import datetime

json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)

Then use JSON serialization as you always do, this time with datetime being serialized as isoformat.

json.dumps({'created':datetime.datetime.now()})

Resulting in: '{"created": "2015-08-26T14:21:31.853855"}'

See more details and some words of caution at: Stack Overflow: JSON datetime between Python and JavaScript

Answer 13

The simplest way to do this is to change the part of the dict that is in datetime format to isoformat. That value will effectively be a string in isoformat which json is OK with.

v_dict = version.dict()
v_dict['created_at'] = v_dict['created_at'].isoformat()

Answer 14

Here is my full solution for converting datetime to JSON and back...

import calendar, datetime, json

def outputJSON(obj):
    """Default JSON serializer."""

    if isinstance(obj, datetime.datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()

        return obj.strftime('%Y-%m-%d %H:%M:%S.%f')
    return str(obj)

def inputJSON(obj):
    newDic = {}

    for key in obj:
        try:
            if float(key) == int(float(key)):
                newKey = int(key)
            else:
                newKey = float(key)

            newDic[newKey] = obj[key]
            continue
        except ValueError:
            pass

        try:
            newDic[str(key)] = datetime.datetime.strptime(obj[key], '%Y-%m-%d %H:%M:%S.%f')
            continue
        except TypeError:
            pass

        newDic[str(key)] = obj[key]

    return newDic

x = {'Date': datetime.datetime.utcnow(), 34: 89.9, 12.3: 90, 45: 67, 'Extra': 6}

print x

with open('my_dict.json', 'w') as fp:
    json.dump(x, fp, default=outputJSON)

with open('my_dict.json') as f:
    my_dict = json.load(f, object_hook=inputJSON)

print my_dict

Output

{'Date': datetime.datetime(2013, 11, 8, 2, 30, 56, 479727), 34: 89.9, 45: 67, 12.3: 90, 'Extra': 6}
{'Date': datetime.datetime(2013, 11, 8, 2, 30, 56, 479727), 34: 89.9, 45: 67, 12.3: 90, 'Extra': 6}

JSON File

{"Date": "2013-11-08 02:30:56.479727", "34": 89.9, "45": 67, "12.3": 90, "Extra": 6}

This has enabled me to import and export strings, ints, floats and datetime objects. It shouldn't be to hard to extend for other types.

Answer 15

I had encountered the same problem when externalizing a Django model object to dump as JSON.

Here is how you can solve it.

def externalize(model_obj):
  keys = model_obj._meta.get_all_field_names()
  data = {}
  for key in keys:
    if key == 'date_time':
      date_time_obj = getattr(model_obj, key)
      data[key] = date_time_obj.strftime("%A %d. %B %Y")
    else:
      data[key] = getattr(model_obj, key)
  return data

Answer 16

Here is my solution:

import json

class DatetimeEncoder(json.JSONEncoder):
    def default(self, obj):
        try:
            return super().default(obj)
        except TypeError:
            return str(obj)

Then you can use it like that:

json.dumps(dictionary, cls=DatetimeEncoder)

Answer 17

For others who do not need or want to use the pymongo library for this, you can achieve datetime JSON conversion easily with this small snippet:

def default(obj):
    """Default JSON serializer."""
    import calendar, datetime

    if isinstance(obj, datetime.datetime):
        if obj.utcoffset() is not None:
            obj = obj - obj.utcoffset()
        millis = int(
            calendar.timegm(obj.timetuple()) * 1000 +
            obj.microsecond / 1000
        )
        return millis
    raise TypeError('Not sure how to serialize %s' % (obj,))

Then use it like so:

import datetime, json
print json.dumps(datetime.datetime.now(), default=default)

Output:

'1365091796124'

Answer 18

You have to supply a custom encoder class with the cls parameter of json.dumps. To quote from the documentation:

>>> import json
>>> class ComplexEncoder(json.JSONEncoder):
...     def default(self, obj):
...         if isinstance(obj, complex):
...             return [obj.real, obj.imag]
...         return json.JSONEncoder.default(self, obj)
...
>>> dumps(2 + 1j, cls=ComplexEncoder)
'[2.0, 1.0]'
>>> ComplexEncoder().encode(2 + 1j)
'[2.0, 1.0]'
>>> list(ComplexEncoder().iterencode(2 + 1j))
['[', '2.0', ', ', '1.0', ']']

This uses complex numbers as the example, but you can just as easily create a class to encode dates (except I think JSON is a little fuzzy about dates).

Answer 19

I usually use orjson. Not only because of its tremendous performance, but also for its great (RFC-3339 compliant) support of datetime:

import orjson # via pip3 install orjson
from datetime import datetime

data = {"created_at": datetime(2022, 3, 1)}

orjson.dumps(data) # returns b'{"created_at":"2022-03-01T00:00:00"}'

If you would like to use datetime.datetime objects without a tzinfo as UTC you can add the related option:

orjson.dumps(data, option=orjson.OPT_NAIVE_UTC) # returns b'{"created_at":"2022-03-01T00:00:00+00:00"}'

Answer 20

My quick & dirty JSON dump that eats dates and everything:

json.dumps(my_dictionary, indent=4, sort_keys=True, default=str)

default is a function applied to objects that aren't serializable.
In this case it's str, so it just converts everything it doesn't know to strings. Which is great for serialization but not so great when deserializing (hence the "quick & dirty") as anything might have been string-ified without warning, e.g. a function or numpy array.

Answer 21

Updated for 2018

The original answer accommodated the way MongoDB "date" fields were represented as:

{"$date": 1506816000000}

If you want a generic Python solution for serializing datetime to json, check out @jjmontes' answer for a quick solution which requires no dependencies.

---

As you are using mongoengine (per comments) and pymongo is a dependency, pymongo has built-in utilities to help with json serialization:
http://api.mongodb.org/python/1.10.1/api/bson/json_util.html

Example usage (serialization):

from bson import json_util
import json

json.dumps(anObject, default=json_util.default)

Example usage (deserialization):

json.loads(aJsonString, object_hook=json_util.object_hook)

---

Django

Django provides a native DjangoJSONEncoder serializer that deals with this kind of properly.

See https://docs.djangoproject.com/en/dev/topics/serialization/#djangojsonencoder

from django.core.serializers.json import DjangoJSONEncoder

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  cls=DjangoJSONEncoder
)

One difference I've noticed between DjangoJSONEncoder and using a custom default like this:

import datetime
import json

def default(o):
    if isinstance(o, (datetime.date, datetime.datetime)):
        return o.isoformat()

return json.dumps(
  item,
  sort_keys=True,
  indent=1,
  default=default
)

Is that Django strips a bit of the data:

 "last_login": "2018-08-03T10:51:42.990", # DjangoJSONEncoder 
 "last_login": "2018-08-03T10:51:42.990239", # default

So, you may need to be careful about that in some cases.

Answer 22

Actually it is quite simple. If you need to often serialize dates, then work with them as strings. You can easily convert them back as datetime objects if needed.

If you need to work mostly as datetime objects, then convert them as strings before serializing.

import json, datetime

date = str(datetime.datetime.now())
print(json.dumps(date))
"2018-12-01 15:44:34.409085"
print(type(date))
<class 'str'>

datetime_obj = datetime.datetime.strptime(date, '%Y-%m-%d %H:%M:%S.%f')
print(datetime_obj)
2018-12-01 15:44:34.409085
print(type(datetime_obj))
<class 'datetime.datetime'>

As you can see, the output is the same in both cases. Only the type is different.

Answer 23

Try this one with an example to parse it:

#!/usr/bin/env python

import datetime
import json

import dateutil.parser  # pip install python-dateutil


class JSONEncoder(json.JSONEncoder):

    def default(self, obj):
        if isinstance(obj, datetime.datetime):
            return obj.isoformat()
        return super(JSONEncoder, self).default(obj)


def test():
    dts = [
        datetime.datetime.now(),
        datetime.datetime.now(datetime.timezone(-datetime.timedelta(hours=4))),
        datetime.datetime.utcnow(),
        datetime.datetime.now(datetime.timezone.utc),
    ]
    for dt in dts:
        dt_isoformat = json.loads(json.dumps(dt, cls=JSONEncoder))
        dt_parsed = dateutil.parser.parse(dt_isoformat)
        assert dt == dt_parsed
        print(f'{dt}, {dt_isoformat}, {dt_parsed}')
        # 2018-07-22 02:22:42.910637, 2018-07-22T02:22:42.910637, 2018-07-22 02:22:42.910637
        # 2018-07-22 02:22:42.910643-04:00, 2018-07-22T02:22:42.910643-04:00, 2018-07-22 02:22:42.910643-04:00
        # 2018-07-22 06:22:42.910645, 2018-07-22T06:22:42.910645, 2018-07-22 06:22:42.910645
        # 2018-07-22 06:22:42.910646+00:00, 2018-07-22T06:22:42.910646+00:00, 2018-07-22 06:22:42.910646+00:00


if __name__ == '__main__':
    test()

Answer 24

Convert the date to a string

sample['somedate'] = str( datetime.utcnow() )

Answer 25

Building on other answers, a simple solution based on a specific serializer that just converts datetime.datetime and datetime.date objects to strings.

from datetime import date, datetime

def json_serial(obj):
    """JSON serializer for objects not serializable by default json code"""

    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    raise TypeError ("Type %s not serializable" % type(obj))

As seen, the code just checks to find out if object is of class datetime.datetime or datetime.date, and then uses .isoformat() to produce a serialized version of it, according to ISO 8601 format, YYYY-MM-DDTHH:MM:SS (which is easily decoded by JavaScript). If more complex serialized representations are sought, other code could be used instead of str() (see other answers to this question for examples). The code ends by raising an exception, to deal with the case it is called with a non-serializable type.

This json_serial function can be used as follows:

from datetime import datetime
from json import dumps

print dumps(datetime.now(), default=json_serial)

The details about how the default parameter to json.dumps works can be found in Section Basic Usage of the json module documentation.

Answer 26

I may not 100% correct but, this is the simple way to do serialize

#!/usr/bin/python
import datetime,json

sampledict = {}
sampledict['a'] = "some string"
sampledict['b'] = datetime.datetime.now()

print sampledict   # output : {'a': 'some string', 'b': datetime.datetime(2017, 4, 15, 5, 15, 34, 652996)}

#print json.dumps(sampledict)

'''
output : 

Traceback (most recent call last):
  File "./jsonencodedecode.py", line 10, in <module>
    print json.dumps(sampledict)
  File "/usr/lib/python2.7/json/__init__.py", line 244, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.datetime(2017, 4, 15, 5, 16, 17, 435706) is not JSON serializable


'''

sampledict['b'] = datetime.datetime.now().strftime("%B %d, %Y %H:%M %p")

afterdump = json.dumps(sampledict)

print afterdump  #output : {"a": "some string", "b": "April 15, 2017 05:18 AM"}

print type(afterdump) #<type 'str'>


afterloads = json.loads(afterdump) 

print afterloads # output : {u'a': u'some string', u'b': u'April 15, 2017 05:18 AM'}


print type(afterloads) # output :<type 'dict'> 

Answer 27

A quick fix if you want your own formatting

for key,val in sample.items():
    if isinstance(val, datetime):
        sample[key] = '{:%Y-%m-%d %H:%M:%S}'.format(val) #you can add different formating here
json.dumps(sample)

Answer 28

Convert the date to string

date = str(datetime.datetime(somedatetimehere)) 

Answer 29

Here is a simple solution to over come "datetime not JSON serializable" problem.

enco = lambda obj: (
    obj.isoformat()
    if isinstance(obj, datetime.datetime)
    or isinstance(obj, datetime.date)
    else None
)

json.dumps({'date': datetime.datetime.now()}, default=enco)

Output:-> {"date": "2015-12-16T04:48:20.024609"}

Answer 30

I got the same error message while writing the serialize decorator inside a Class with sqlalchemy. So instead of :

Class Puppy(Base):
    ...
    @property
    def serialize(self):
        return { 'id':self.id,
                 'date_birth':self.date_birth,
                  ...
                }

I simply borrowed jgbarah's idea of using isoformat() and appended the original value with isoformat(), so that it now looks like:

                  ...
                 'date_birth':self.date_birth.isoformat(),
                  ...