CODEWITHSUNDEEP
pythonjsonunicodeencode
seyed_vahid
seyed_vahid

Reputation: 49

'UCS-2' codec can't encode characters in position 61-61

When I run my Python code and print(item), I get the following errors:

UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 61-61: Non-BMP character not supported in Tk

Here is my code:

def getUserFollowers(self, usernameId, maxid = ''):
    if maxid == '':
        return self.SendRequest('friendships/'+ str(usernameId) +'/followers/?rank_token='+ self.rank_token,l=2)
    else:
        return self.SendRequest('friendships/'+ str(usernameId) +'/followers/?rank_token='+ self.rank_token + '&max_id='+ str(maxid))
def getTotalFollowers(self,usernameId):
    followers = []
    next_max_id = ''
    while 1:
        self.getUserFollowers(usernameId,next_max_id)
        temp = self.LastJson

        for item in temp["users"]:
            print(item)
            followers.append(item)

        if temp["big_list"] == False:
            return followers            
        next_max_id = temp["next_max_id"]

How can I fix this?

Upvotes: 2

Views: 3882

Answers (1)

Serge Ballesta
Serge Ballesta

Reputation: 148975

Hard to guess without knowing the content of temp["users"], but the error indicates that it contains non BMP unicode characters like for example emoji.

If you try to display that in IDLE, you immediately get that kind of error. Simple example to reproduce (on IDLE for Python 3.5):

>>> t = "ab \U0001F600 cd"
>>> print(t)
Traceback (most recent call last):
  File "<pyshell#5>", line 1, in <module>
    print(t)
UnicodeEncodeError: 'UCS-2' codec can't encode characters in position 3-3: Non-BMP character not supported in Tk

(\U0001F600 represents the unicode character U+1F600 grinning face)

The error is indeed caused by Tk not supporting unicode characters with code greater than FFFF. A simple workaround is the filter them out of your string:

def BMP(s):
    return "".join((i if ord(i) < 10000 else '\ufffd' for i in s))

'\ufffd' is the Python representation for the unicode U+FFFD REPLACEMENT CHARACTER.

My example becomes: 

>>> t = "ab \U0001F600 cd"
>>> print(BMP(t))
ab � cd

So your code would become:

for item in temp["users"]:
    print(BMP(item))
    followers.append(item)

Upvotes: 5

Related Questions