CODEWITHSUNDEEP
pythonfile
Dr. UK
Dr. UK

Reputation: 73

Deleting lines as you go in a big txt file

I have red all questions in stackoverflow about it, they all say make it a list or put everything in another txt file etc. I can't make them because my txt file is bigger than 1gb, I can only read that file with for loop.

I tried to make:

f = r.read()

and go outside for 3 hours.

When I come back it was still reading it.

So then I changed it to: 

with open("wordlist.txt") as f:
        for sat in f:
            try:
                dene(sat.strip())

But then when the program fails I have to run the whole wordlist again. So what I want to do looks like:

with open("wordlist.txt") as f:
        for sat in f:
            try:
                dene(sat.strip())
                f.delete(sat)

So if the program fails and I open it again it will not start from first.

Upvotes: 0

Views: 46

Answers (1)

Ohumeronen
Ohumeronen

Reputation: 2086

I think you can't delete the lines on the fly. You can however save the linenumber in which you have last been. When your script fails, you can simply do the following:

[f.next() for _ in range(lines_count)]

Where lines_count was previously stored in a textfile. It is the number of lines that have already been processed. So basically you move all the way down to the line where you have last been and go on from there.

Upvotes: 2

Related Questions