CODEWITHSUNDEEP
python-2.7io
a06e
a06e

Reputation: 20774

Read one file at two different positions simultaneously?

I have a very long file, and I need to read it by processing two different lines at a time. For example, I would like to do something like this:

with open('test.txt', 'r') as f1, open('test.txt', 'r') as f2:
    for l1 in f1:
        for l2 in f2:
            process(l1, l2)

where process is some processing function, and test.txt is a huge file, so I can't load it all into memory at once.

The code above doesn't work. When l2 reaches the end of the file, both loops end, as if it only keeps track of a single position in the file, instead of two. How can I do what I want?

Upvotes: 0

Views: 79

Answers (1)

swenzel
swenzel

Reputation: 7173

f2 will be already consumed by the first run once your outer loop reaches the second one.
You'll have to reset it with f2.seek(0) after the end of the inner loop.

Try this:

with open('test.txt', 'r') as f1, open('test.txt', 'r') as f2:
    for l1 in f1:
        for l2 in f2:
            process(l1, l2)
        f2.seek(0)

Upvotes: 2

Related Questions