CODEWITHSUNDEEP
pythonunicode
Rolando
Rolando

Reputation: 62704

How to replace unicode characters in string with something else python?

I have a string that I got from reading a HTML webpage with bullets that have a symbol like "•" because of the bulleted list. Note that the text is an HTML source from a webpage using Python 2.7's urllib2.read(webaddress).

I know the unicode character for the bullet character as U+2022, but how do I actually replace that unicode character with something else?

I tried doing str.replace("•", "something")

but it does not appear to work... how do I do this?

Upvotes: 57

Views: 177297

Answers (7)

Ayushman Verma
Ayushman Verma

Reputation: 47

str1 = "This is Python\u500cPool"

Encode the string to ASCII and replace all the utf-8 characters with '?'.

str1 = str1.encode("ascii", "replace")

Decode the byte stream to string.

str1 = str1.decode(encoding="utf-8", errors="ignore")

Replace the question mark with the desired character.

str1 = str1.replace("?"," ")

Upvotes: -1

Khánh Pluto
Khánh Pluto

Reputation: 15

If you want to remove all \u character. Code below for you

def replace_unicode_character(self, content: str):
    content = content.encode('utf-8')
    if "\\x80" in str(content):
        count_unicode = 0
        i = 0
        while i < len(content):
            if "\\x" in str(content[i:i + 1]):
                if count_unicode % 3 == 0:
                    content = content[:i] + b'\x80\x80\x80' + content[i + 3:]
                i += 2
                count_unicode += 1
            i += 1

        content = content.replace(b'\x80\x80\x80', b'')
    return content.decode('utf-8')

Upvotes: -2

Rahul Kumar Gupta
Rahul Kumar Gupta

Reputation: 241

Try this one.

you will get the output in a normal string

str.encode().decode('unicode-escape')

and after that, you can perform any replacement.

str.replace('•','something')

Upvotes: 11

Mafketel
Mafketel

Reputation: 117

Funny the answer is hidden in among the answers.

str.replace("•", "something")

would work if you use the right semantics.

str.replace(u"\u2022","something")

works wonders ;) , thnx to RParadox for the hint.

Upvotes: -2

RParadox
RParadox

Reputation: 6891

Encode string as unicode.

>>> special = u"\u2022"
>>> abc = u'ABC•def'
>>> abc.replace(special,'X')
u'ABCXdef'

Upvotes: 16

David
David

Reputation: 6561

import re
regex = re.compile("u'2022'",re.UNICODE)
newstring = re.sub(regex, something, yourstring, <optional flags>)

Upvotes: 3

Fred Foo
Fred Foo

Reputation: 363818

  1. Decode the string to Unicode. Assuming it's UTF-8-encoded:

    str.decode("utf-8")

  2. Call the replace method and be sure to pass it a Unicode string as its first argument:

    str.decode("utf-8").replace(u"\u2022", "*")

  3. Encode back to UTF-8, if needed:

    str.decode("utf-8").replace(u"\u2022", "*").encode("utf-8")

(Fortunately, Python 3 puts a stop to this mess. Step 3 should really only be performed just prior to I/O. Also, mind you that calling a string str shadows the built-in type str.)

Upvotes: 85

Related Questions