Replace string in a specific line using python

Question

I'm writing a python script to replace strings from a each text file in a directory with a specific extension (.seq). The strings replaced should only be from the second line of each file, and the output is a new subdirectory (call it clean) with the same file names as the original files, but with a *.clean suffix. The output file contains exactly the same text as the original, but with the strings replaced. I need to replace all these strings: 'K','Y','W','M','R','S' with 'N'.

This is what I've come up with after googling. It's very messy (2nd week of programming), and it stops at copying the files into the clean directory without replacing anything. I'd really appreciate any help.

Thanks before!

import os, shutil

os.mkdir('clean')

for file in os.listdir(os.getcwd()):
    if file.find('.seq') != -1:
        shutil.copy(file, 'clean')

os.chdir('clean')

for subdir, dirs, files in os.walk(os.getcwd()):
    for file in files:
        f = open(file, 'r')
        for line in f.read():
            if line.__contains__('>'): #indicator for the first line. the first line always starts with '>'. It's a FASTA file, if you've worked with dna/protein before.
                pass
            else:
                line.replace('M', 'N')
                line.replace('K', 'N')
                line.replace('Y', 'N')
                line.replace('W', 'N')
                line.replace('R', 'N')
                line.replace('S', 'N')

Answer 1

some notes:

1. string.replace and re.sub are not in-place so you should be assigning the return value back to your variable.
2. glob.glob is better for finding files in a directory matching a defined pattern...
3. maybe you should be checking if the directory already exists before creating it (I just assumed this, this could not be your desired behavior)
4. the with statement takes care of closing the file in a safe way. if you don't want to use it you have to use try finally.
5. in your example you where forgetting to put the sufix *.clean ;)
6. you where not actually writing the files, you could do it like i did in my example or use fileinput module (which until today i did not know)

here's my example:

import re
import os
import glob

source_dir=os.getcwd()
target_dir="clean"
source_files = [fname for fname in glob.glob(os.path.join(source_dir,"*.seq"))]

# check if target directory exists... if not, create it.
if not os.path.exists(target_dir):
    os.makedirs(target_dir)

for source_file in source_files:
   target_file = os.path.join(target_dir,os.path.basename(source_file)+".clean")
   with open(source_file,'r') as sfile:
      with open(target_file,'w') as tfile:
         lines = sfile.readlines()
         # do the replacement in the second line.
         # (remember that arrays are zero indexed)
         lines[1]=re.sub("K|Y|W|M|R|S",'N',lines[1])
         tfile.writelines(lines)

print "DONE"

hope it helps.

Answer 2

you need to allocate the result of the replacement back to "line" variable

line=line.replace('M', 'N')

you can also use the module fileinput for inplace edit

import os, shutil,fileinput
if not os.path.exists('clean'):
    os.mkdir('clean')

for file in os.listdir("."):
    if file.endswith(".seq"):
        shutil.copy(file, 'clean')

os.chdir('clean')

for subdir, dirs, files in os.walk("."):
    for file in files:
        f = fileinput.FileInput(file,inplace=0)
        for n,line in enumerate(f):
            if line.lstrip().startswith('>'):
                pass
            elif n==1: #replace 2nd line
                for repl in ["M","K","Y","W","R","S"]:
                    line=line.replace(ch, 'N')
            print line.rstrip()
        f.close()

change inplace=0 to inplace=1 for in place editing of your files.

Answer 3

Here are some general hints:

1. Don't use find for checking the file extension (e.g., this would also match "file1.seqdata.xls"). At least use file.endswith('seq'), or, better yet, os.path.splitext(file)[1]

2. Actually, don't do that altogether. This is what you want:

   import glob
   seq_files = glob.glob("*.seq")
   

3. Don't copy the files, it's much easier to use just one loop:

   for filename in seq_files:
       in_file = open(filename)
       out_file = open(os.path.join("clean", filename), "w")
       # now read lines from in_file and write lines to out_file
   

4. Don't use line.__contains__('>'). What you mean is

   if '>' in line:
   

   (which will call __contains__ internally). But actually, you want to know wether the line starts with a `">", not if there's one somewhere within the line, be it at the beginning or not. So the better way would be this:

   if line.startswith(">"):
   

   I'm not familiar with your file type; if the ">" check really is just for determining the first line, there's better ways to do that.

5. You don't need the if block (you just pass). It's cleaner to write

   if not something:
       do_things()
   other_stuff()
   

   instead of

   if something:
       pass
   else:
       do_things()
   other_stuff()
   

Have fun learning Python!

Answer 4

You should replace line.replace('M', 'N') with line=line.replace('M', 'N'). replace returns a copy of the original string with the relevant substrings replaced.

An even better way (IMO) is to use re.

import re

line="ABCDEFGHIJKLMNOPQRSTUVWXYZ"
line=re.sub("K|Y|W|M|R|S",'N',line)
print line 

Answer 5

line.replace is not a mutator, it leaves the original string unchanged and returns a new string with the replacements made. You'll need to change your code to line = line.replace('R', 'N'), etc.

I think you also want to add a break statement at the end of your else clause, so that you don't iterate over the entire file, but stop after having processed line 2.

Lastly, you'll need to actually write the file out containing your changes. So far, you are just reading the file and updating the line in your program variable 'line'. You need to actually create an output file as well, to which you will write the modified lines.