CODEWITHSUNDEEP
python
Iram Khan
Iram Khan

Reputation: 37

Python delete line/lines from file without modifying existing contents

I have to delete string or list of strings based on user input from a file. I referred the below link and things are working fine.

Deleting a specific line in a file (python)

However, the above approach reads the existing file contents in memory and if the line to delete is not found writes it back in the same file. This approach is not suitable if we dealing with files with huge amount of confidential data.

All i wish to know is, Is there a better way to do the same thing.

  valid_List=["10.1.2.3","10.2.3.4","10.2.4.5","10.2.3.7"]
  filename="abc.txt"
  for i in valid_List:
    f = open(filename,"r")
    lines = f.readlines()
    f.close()
    f = open(filename,"w")
    for line in lines:
      if line!=i+" "+ "ok"+"\n":
        #print("Writing ip not to be deleted")
        f.write(line)
      else:
        print(i," Deleted")
        user_response.append(i+" Deleted")
        logger.info('Response returned to user%s',user_response)
    f.close()

Upvotes: 0

Views: 856

Answers (3)

Hani Yassine
Hani Yassine

Reputation: 31

i created this script basically you put bunch of lines strings in a list if any of them was found it get deleted and it works on batch so it open multiple files you input the number of files obviously it only for personal use not users because it doesnt have input check and the files need to be in the same dir as the script:

n=int(input('enter the number of files:'))
for i in range (1,n):
    f = open(f"{i}.txt","r")
    lines = f.readlines()
    f.close()
    f = open(f"{i}.txt","w")
    strings_to_remove=['Edited at','test']
    for line in lines:
        if line.strip() not in strings_to_remove:
            f.write(line)
    f.close()

Upvotes: 1

Nils Werner
Nils Werner

Reputation: 36849

You can read and write to two different files and do the operation elementwise.

Afterwards you replace the inputfile with the outputfile

import shutil

valid_List = ["10.1.2.3", "10.2.3.4", "10.2.4.5", "10.2.3.7"]
filename = "abc.txt"
outfile = "outfile.txt"

with open(filename, "r") as f:
    with open(outfile, "w") as o:
        for line in f:
            if all([line != "%s ok\n" % i for i in valid_List]):
                o.write(line)
            else:
                print("%s Deleted" % line.strip())

shutil.move(outfile, filename)

Caveat This uses the a fixed filename for output, which might cause collisions when you run the program multiple times in parallel. If you use this atomic save recipe you can simplify the code to

valid_List = ["10.1.2.3", "10.2.3.4", "10.2.4.5", "10.2.3.7"]
filename = "abc.txt"

with atomic_open(filename, "w") as o:
    with open(filename, "r") as f:
        for line in f:
            if all([line != "%s ok\n" % i for i in valid_List]):
                o.write(line)
            else:
                print("%s Deleted" % line.strip())

This will automatically choose a temporary file (collision-free) for you and replace the input file with the output file on completion.

Also you will notice that I have replaced your outer loop (opening files once for each entry in valid_list) with an all() statement. This saves you a lot of overhead, too.

Upvotes: 4

Abdul Fatir
Abdul Fatir

Reputation: 6357

You're opening and closing the huge file multiple times, once for each element in valid_List. You should instead open the file just once and check if any line of file matches with your valid_List.

Try like this (the code is not tested but it should work): 

valid_List=["10.1.2.3","10.2.3.4","10.2.4.5","10.2.3.7"]
filename="abc.txt"

f = open(filename,"r")
lines = f.readlines()
f.close()

f = open(filename,"w")
for line in lines:
    flag = True
    deleted = ''
    for i in valid_List:
        if line == i+" "+ "ok"+"\n":
            flag = False
            deleted = i
            break
    if flag:
        #print("Writing ip not to be deleted")
        f.write(line)
    else:
        print(deleted," Deleted")
f.close()

EDIT
Added check for not-found IPs. 

valid_List=["10.1.2.3","10.2.3.4","10.2.4.5","10.2.3.7"]
filename="abc.txt"

if_found = [False for v in valid_List]

f = open(filename,"r")
lines = f.readlines()
f.close()

f = open(filename,"w")
for line in lines:
    flag = True
    deleted = ''
    for _,i in enumerate(valid_List):
        if line == i+" "+ "ok"+"\n":
            flag = False
            if_found[_] = True
            deleted = i
            break
    if flag:
        #print("Writing ip not to be deleted")
        f.write(line)
    else:
        print(deleted," Deleted")
f.close()

for _,i in enumerate(if_found):
    if not i:
        print(valid_List[_]," Not Found")

Upvotes: 1

Related Questions