CODEWITHSUNDEEP
pythonunicode
user2377057
user2377057

Reputation: 183

Python Check for unicode files

I'm very new to python and I have a mix of both ansi and unicode (utf-16-le) text based files in a series of directories. I've got some code which reads the text files okay until it hits a unicode file which at the mo, I've written in the code to skip. . I'm wondering if there's anyway I can get python to run a 

with codecs.open

type of thing when it hits a unicode file as part of one prog? With my currrent level of python experience, the only way I could see of doing this is to write two separate progs; one to process the ANSI stuff and one for the Unicode.

Thanks in advance for any help you can provide

Upvotes: 1

Views: 129

Answers (2)

thkang
thkang

Reputation: 11543

use unicode by default(which is a good programming discipline) and switch to ansi only if necessary.

import codecs

def opener(filename):
    try:
        f = codecs.open(filename, encoding='utf-8')
    except UnicodeError:
        f = open(filename)
    return f

Upvotes: 1

Jordan Jambazov
Jordan Jambazov

Reputation: 3620

Just open all files using UTF-8.

f = codecs.open(file_name, "r", "utf-8")

Upvotes: 0

Related Questions