CODEWITHSUNDEEP
pythonunicodeutf-8
Henry Thornton
Henry Thornton

Reputation: 4577

Python Convert Unicode-Hex utf-8 strings to Unicode strings

Have s = u'Gaga\xe2\x80\x99s' but need to convert to t = u'Gaga\u2019s'

How can this be best achieved?

Upvotes: 5

Views: 7528

Answers (3)

Mark Tolonen
Mark Tolonen

Reputation: 178389

Where ever you decoded the original string, it was likely decoded with latin-1 or a close relative. Since latin-1 is the first 256 codepoints of Unicode, this works:

>>> s = u'Gaga\xe2\x80\x99s'
>>> s.encode('latin-1').decode('utf8')
u'Gaga\u2019s'

Upvotes: 7

unutbu
unutbu

Reputation: 880887

s = u'Gaga\xe2\x80\x99s'
t = u'Gaga\u2019s'
x = s.encode('raw-unicode-escape').decode('utf-8')
assert x==t

print(x)

yields

Gaga’s

Upvotes: 7

rocksportrocker
rocksportrocker

Reputation: 7429

import codecs

s = u"Gaga\xe2\x80\x99s"
s_as_str = codecs.charmap_encode(s)[0]
t = unicode(s_as_str, "utf-8")
print t

prints

u'Gaga\u2019s'

Upvotes: 2

Related Questions