CODEWITHSUNDEEP
pythonpython-2.7
CoXier
CoXier

Reputation: 2663

Python UnicodeEncodeError: 'ascii' codec can't encode characters

There is a json data which contains some Chinese characters.

{
  "font_size": "47",
  "sentences": [
    "你好",
    "sample sentence1",
    "sample sentence2",
    "sample sentence3",
    "sample sentence4",
    "sample sentence5",
    "sample sentence6",
    "sample sentence7",
    "sample sentence8",
    "sample sentence9"
  ]
}

I create a Flask app and use it to receive above json data. I use below curl command to post data.

curl -X POST \
  http://0.0.0.0:5000/ \
  -H 'Cache-Control: no-cache' \
  -H 'Content-Type: application/json;charset=UTF-8' \
  -H 'Postman-Token: af380f7a-42a8-cfbb-9177-74bb348ce5ed' \
  -d '{
  "font_size": "47",
  "sentences": [
    "你好",
    "sample sentence1",
    "sample sentence2",
    "sample sentence3",
    "sample sentence4",
    "sample sentence5",
    "sample sentence6",
    "sample sentence7",
    "sample sentence8",
    "sample sentence9"
  ]
}'

After I receive json data from request.data, I convert it to json, in fact request.data is str.

json_data = json.loads(request.data)

Then I want to format a string with json_data.

subtitles.format(**json_data)

I got an error.

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)

How to solve it? Thanks in advance.

EDIT

subtitles is read from file.

subtitles_file = "{path}/templates/sorry/subtitles.ass".format(path=root_path)
with open(subtitles_file, 'r') as file:
     subtitles = file.read()

EDIT Python 2 or Python 3

I'm using python 2 and this error occurs. However Python 3 can automatically handle this. So enjoy Python 3.

Upvotes: 3

Views: 15579

Answers (1)

abarnert
abarnert

Reputation: 366003

In Python 2, when you open and read a file, what you get is a regular str, not a unicode.

Meanwhile, even if request.data is a str rather than a unicode, if any of the strings in it are non-ASCII, json_data will contain unicode.

So, when you do subtitles.format, that's going to try to encode each unicode using your default encoding—which, if you haven't done anything, is ASCII. Which will give exactly this error.

The simplest fix is to change subtitles to a unicode. Like this:

with open(subtitles_file, 'r') as file:
    subtitles = file.read().decode('utf-8')

… or:

with codecs.open(subtitles_file, 'r', 'utf-8') as file:
    subtitles = file.read()

(I'm guessing that you want UTF-8; if your files are in some other encoding, obviously use that instead.)

Upvotes: 2

Related Questions