CODEWITHSUNDEEP
python
socks_swerve
socks_swerve

Reputation: 143

Make the readline method of Python recognize both end-of-line variations?

I am writing a Python file that needs to read in several files of different types. I am reading the files in line by line with the traditional for line in f after using f = open("file.txt", "r").

This doesn't seem to be working for all files. My guess is some files end with different encodings (such as \r\n versus just \r). I can read the whole file in and do a string split on \r, but that is hugely costly and I'd rather not. Is there a way to make the readline method of Python recognize both end-of-line variations?

Upvotes: 12

Views: 10288

Answers (2)

Kos
Kos

Reputation: 72319

You can try to use a generator approach to read the lines by yourself and ignore any EOL characters:

def readlines(f):
    line = []
    while True:
        s = f.read(1)
        if len(s) == 0:
            if len(line) > 0:
                yield line
            return
        if s in ('\r','\n'):
            if len(line) > 0:
                yield line
            line = []
        else:
            line.append(s)

for line in readlines(yourfile):
    # ...

Upvotes: 0

bgporter
bgporter

Reputation: 36564

Use the universal newline support -- see http://docs.python.org/library/functions.html#open

In addition to the standard fopen() values mode may be 'U' or 'rU'. Python is usually built with universal newline support; supplying 'U' opens the file as a text file, but lines may be terminated by any of the following: the Unix end-of-line convention '\n', the Macintosh convention '\r', or the Windows convention '\r\n'. All of these external representations are seen as '\n' by the Python program. If Python is built without universal newline support a mode with 'U' is the same as normal text mode. Note that file objects so opened also have an attribute called newlines which has a value of None (if no newlines have yet been seen), '\n', '\r', '\r\n', or a tuple containing all the newline types seen.

Upvotes: 18

Related Questions