CODEWITHSUNDEEP
pythonjsonpython-3.xpython-3.5
eatonphil
eatonphil

Reputation: 13722

Json dumping bytes fails in Python 3

I am sending binary data in a post request as part of a request. I have a dictionary that looks like this:

data = {"foo": "bar", "bar": b'foo'}

When I try to json.dumps this dictionary, I get the following exception:

TypeError: b'foo' is not JSON serializable

This worked fine in Python 2.7. What do I have to do to json encode this data?

Upvotes: 6

Views: 7105

Answers (3)

David Bau
David Bau

Reputation: 3959

The cls argument of json.dump lets you specify more encoding rules. For handling bytes, you can say:

import json

class ByteEncoder(json.JSONEncoder):
    def default(self, x):
        return x.decode('utf-8') if isinstance(x, bytes) else super().default(x)

print(json.dumps({'a': 'aa', 'b': b'bb'}, indent=1, cls=ByteEncoder))

Upvotes: 0

yasir khatri
yasir khatri

Reputation: 101

With Json module you cannot dump bytes. A suitable alternative is to use Simple Json Module.

To Install Simple json:

pip3 install simplejson

Code:

import simplejson as json
data = {"foo": "bar", "bar": b"foo"}
json.dumps(data)

Hopefully you won't get the error now!

Upvotes: 4

Michael Recachinas
Michael Recachinas

Reputation: 2749

In Python 3, they removed byte support in json. (Source: https://bugs.python.org/issue10976).

A possible workaround is:

import json

data = {"foo": "bar", "bar": b"foo"}

# decode the `byte` into a unicode `str`
data["bar"] = data["bar"].decode("utf8")

# `data` now contains
#
#   {'bar': 'foo', 'foo': 'bar'}
#
# `json_encoded_data` contains
#
#   '{"bar": "foo", "foo": "bar"}'
#
json_encoded_data = json.dumps(data)

# `json_decoded_data` contains
#
#   {'bar': 'foo', 'foo': 'bar'}
#
json_decoded_data = json.loads(data)

# `data` now contains
#
#   {'bar': b'foo', 'foo': 'bar'}
#
data["bar"] = data["bar"].encode("utf8")

If you don't have a constraint of using json, you might consider using bson (Binary JSON):

import bson

data = {"foo": "bar", "bar": b"foo"}

# `bson_encoded_data` contains 
#
#   b'\x1f\x00\x00\x00\x05bar\x00\x03\x00\x00\x00\x00foo\x02foo\x00\x04\x00\x00\x00bar\x00\x00'
#
bson_encoded_data = bson.BSON.encode(data)

# `bson_decoded_data` contains
#
#   {'bar': b'foo', 'foo': 'bar'}
#
bson_decoded_data = bson.BSON.decode(data)

Upvotes: 11

Related Questions