CODEWITHSUNDEEP
pythonpython-2.7python-requests
help please
help please

Reputation: 57

Decoding chinese characters in python requests

I'm using newsapi.org and trying to pull chinese articles. When I print them, it can't seem to read chinese characters. u'\u8fdd\u89c4 shows up instead.

{u'articles': [{u'author': u'chinanews',
                u'description': u'\u8fdd\u89c4 \u539f\u6599\u836f\u5904\u65b9\u836f\u6d41\u5165\u5e02\u573a\u3000\u3000\u5206\u6790\u5404\u5730\u8b66\u65b9\u516c\u5e03\u7684\u5236\u552e\u6709\u6bd2\u6709\u5bb3\u4fdd\u5065\u54c1\u6848\u60c5\u4e0d\u96be\u53d1\u73b0\uff0c\u8fd9\u4e9b\u6709\u5bb3\u4fdd\u5065\u54c1\u7684\u751f\u4ea7\u539f\u6750\u6599\u4e3b\u8981\u6709\u4e24\u5927\u6e90\u5934\u3002',
                u'publishedAt': u'2018-07-14T20:50:00Z',
                u'source': {u'id': None, u'name': u'Chinanews.com'},

My code is 

import requests

url = ('https://newsapi.org/v2/top-headlines?'
       'country=cn&'
       'apiKey=16a6cd0345d84e799600cdb9ead6f05d')

response = requests.get(url)
pprint(response.json())

Upvotes: 0

Views: 2043

Answers (2)

nneonneo
nneonneo

Reputation: 179422

You're on Python 2.7, which uses a unicode type to store Unicode strings (e.g. the strings that are stored in JSON). The repr for the unicode type prints all non-ASCII characters with Unicode escapes (\uXXXX), and that's what you're seeing in the pprint output.

You'll need to print the value separately to see the characters:

>>> print repr(u'\u8fdd\u89c4 \u539f\u6599\u836f')
u'\u8fdd\u89c4 \u539f\u6599\u836f'
>>> print u'\u8fdd\u89c4 \u539f\u6599\u836f'
违规 原料药

Thus, in your case, you might do something like

for article in response.json()['articles']:
    print article['description']

Upvotes: 1

figbeam
figbeam

Reputation: 7176

I managed to get this far:

# A small part of the first part of the string provided:
text = u'\u8fdd\u89c4 \u539f\u6599\u836f' 
print(text)
# Prints 违规 原料药

bytes_var = text.encode('utf-8')
print(bytes_var)
# Prints b'\xe8\xbf\x9d\xe8\xa7\x84 \xe5\x8e\x9f\xe6\x96\x99\xe8\x8d\xaf'

Now, exactly what is the problem with this? It might have to do with versions; I run Python 3.6.5 under win10.

Upvotes: 0

Related Questions