CODEWITHSUNDEEP
pythonpython-2.7unicodeunicode-stringpython-unicode
user2423706
user2423706

Reputation: 169

Unicode to original character in Python

When I use for example,

unicode_string = u"Austro\u002dHungarian_gulden"
unicode_string.encode("ascii", "ignore")

Then it will give this output:'Austro-Hungarian_gulden'

But I am using a txt file which contains set of data as below:

Austria\u002dHungary    Austro\u002dHungarian_gulden
Cocos_\u0028Keeling\u0029_Islands   Australian_dollar
El_Salvador Col\u00f3n_\u0028currency\u0029
Faroe_Islands   Faroese_kr\u00f3na
Georgia_\u0028country\u0029 Georgian_lari

And I have to process this data using regular expressions in Python, so I have created a script as below, but it does not work for replacing Unicode values with appropiate characters in the string.

Likewise

'\u002d' has appropriate character '-'
'\u0028' has appropriate character '('
'\u0029' has appropriate character ')'

Script for processing text file:

import re
import collections

def extract():
    filename = raw_input("Enter file Name:")
    in_file = file(filename,"r")
    out_file = file("Attribute.txt","w+")
    for line in in_file:
        values = line.split("\t")
        if values[1]:
            str1 = ""
            for list in values[1]:
                list = re.sub("[^\Da-z0-9A-Z()]","",list)
                list = list.replace('_',' ')
                out_file.write(list)
                str1 += list
            out_file.write("    ")
        if values[2]:
            str2 = ""
            for list in values[2]:
                list = re.sub("[^\Da-z0-9A-Z\n]"," ",list)
                list = list.replace('"','')
                list = list.replace('_',' ')
                out_file.write(list)
                str2 += list
        s1 = str1.lstrip()
        s1 = str1.rstrip()
        s2 = str2.lstrip()
        s2 = str2.rstrip()
        print s1+s2

Expected output for the given data is:

Austria-Hungary Austro-Hungarian gulden
Cocos (Keeling) Islands Australian dollar
El Salvador Coln (currency)
FaroeIslands    Faroese krna
Georgia (country)   Georgian lari

How can I do it?

Upvotes: 2

Views: 869

Answers (1)

Janne Karila
Janne Karila

Reputation: 25207

Convert the input into Unicode using decode("unicode_escape"), then encode() the output to an encoding of your choice.

>>> r"Austro\u002dHungarian_gulden".decode("unicode_escape")
u'Austro-Hungarian_gulden'

Upvotes: 2

Related Questions