CODEWITHSUNDEEP
jsonpython-3.xunicodeutf-8
Robycool
Robycool

Reputation: 1224

json loads fails with UnicodeEncodeError

The following code gives a UnicodeEncodeError error in a docker container with ubuntu 18:

Python 3.6.9 (default, Apr 18 2020, 01:56:04) 
import json
text = b'["Chauss\\u00e9e de Tubize"]'
test = json.loads(text)
test 
['Chauss\xe9e de Tubize'] # on other server this correctly results into ['Chaussée de Tubize']
print(test)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 8: ordinal not in range(128)

The strange thing is that this code snippet works on a ubuntu server with the same python version (3.6.9). I don't get why the json module is trying to encode to ascii. I thought that since python3 the standard was to use unicode utf-8.

Upvotes: 0

Views: 259

Answers (1)

Giacomo Catenazzi
Giacomo Catenazzi

Reputation: 9523

No, your error is not about JSON or about encoding of Python3. Your error is about your console charset. Your locale has ASCII charset, and so python cannot print your string.

You should convert and specify how to handle encoding errors, also when using print.

And possibly you should install/use a locale which support UTF-8.

Upvotes: 1

Related Questions