CODEWITHSUNDEEP
pythonjsonunicode
scottlima
scottlima

Reputation: 225

How to remove unicode characters from Dictionary data in python

After using request library , am getting below dict in response.json()

{u'xyz': {u'key1': None, u'key2': u'Value2'}}

I want to remove all unicode characters and print only key value pairs without unicode chars

I have tried below method to remove , but it shows malformed string 

>>> import json, ast
>>> c = {u'xyz': {u'key1': None,u'key2': u'Value2'}}
>>> ast.literal_eval(json.dumps(c))

Getting 'ValueError: malformed string'

Any suggestion on how to do it ?

Upvotes: 9

Views: 46494

Answers (4)

RemcoGerlich
RemcoGerlich

Reputation: 31250

I don't really understand why you want this. Your variable is a normal Python dict with normal Unicode strings, and they happen to be printed as u'' to distinguish them from bytestrings, but that shouldn't matter for using them.

If you want to save them as strings to read them as data later, JSON is a fine format for that. So no need to call request's .json() function at all, just use the response's .text attribute -- it's already JSON, after all.

Your try

>>> ast.literal_eval(json.dumps(c))

Fails because you first turn c into JSON again, and then try to parse it as a Python literal. That doesn't work because Python isn't JSON; in particular one has null and the other has None.

So maybe you want to change the Unicode strings into bytestrings? Like by encoding them as UTF8, that might work:

def to_utf8(d):
    if type(d) is dict:
        result = {}
        for key, value in d.items():
            result[to_utf8(key)] = to_utf8(value)
        return result
    elif type(d) is unicode:
        return d.encode('utf8')
    else:
        return d

Or something like that, but I don't know why you would need it.

Upvotes: 1

S.K. Venkat
S.K. Venkat

Reputation: 1807

This snippet will helps you to preserve the data without unicode prefix notation u :

>>> import json
>>> c = {u'xyz': {u'key1': u'Value1',u'key2': u'Value2'}}
>>> print c
{u'xyz': {u'key2': u'Value2', u'key1': u'Value1'}}
>>> d = eval(json.dumps(c))
>>> print d
{'xyz': {'key2': 'Value2', 'key1': 'Value1'}}

json.dumps() will convert the dict to string type and eval() will reverse it.

Note: key1 value has changed from None to 'value1' for testing purpose

Upvotes: 5

Marichyasana
Marichyasana

Reputation: 3154

You can use unicodestring.encode("ascii","replace") 

>>> ustr=u'apple'
>>> ustr
u'apple'
>>> astr=ustr.encode("ascii","replace")
>>> astr
'apple'

Upvotes: 1

Ohad the Lad
Ohad the Lad

Reputation: 1929

Change your None to 'None':

 c = {u'xyz': {u'key1': 'None', u'key2': u'Value2'}}

it is a casting issue - ast likes str's

Also, maybe u want to change all None to empty str or 'None' str... See this thread : Python: most idiomatic way to convert None to empty string? with this code, i've changes the empty string to 'None':

def xstr(s):
    if s is None:
        return 'None'
    return str(s)

Upvotes: 4

Related Questions