CODEWITHSUNDEEP
pythonfileline
Zewei Song
Zewei Song

Reputation: 571

Avoid to write '\n' to the last line of a file in python

I'm writing multiple lines to a new file (could be up to several GB), like this:

for item in record:
    output_pass.write('%s\n' %item)

However, I got a blank line due to the '\n' of my last record, such as:

Start of the file

record111111

reocrd222222

record333333

---a blank line---

End of a file

Since my file is large, I would not want to read the file again. So, is there an easy way to prevent this, or easy way to remove the last '\n' from the file?

My solution:

Thanks for all the help!

I think I will not load the entire file to the memeory, since it may get very huge.

I actually solve this by first write the first record, then write the rest line in a loop. I put '\n' in the front so it won't appear on the last line.

But Jonathan is right. I actually have now problem with the '\n' in the last line, majorly it is my OCD.

Here is my code:

rec_first = parser_fastq.next() #This is just an iterator of my file
output.write('%s' %('>'+rec_first[0].strip('@')))
output.write('\n%s' %(rec_first[1])) #I put '\n' in the front

count = 1

#Write the rest of lines
for rec_fastq in parser_fastq:
    output.write('\n%s' %('>'+rec_fastq[0].strip('@')))
    output.write('\n%s' %(rec_fastq[1]))
    count += 1
    print 'Extracting %ith record in %s ...' %(count, fastq_name) + '\b'*100,

output.close()

print '\n%i records were wrote to %s' % (count, fasta_name)

Upvotes: 7

Views: 12223

Answers (8)

Amir Rachum
Amir Rachum

Reputation: 79635

This should be a simple solution:

for item in record[:-1]:
    output_pass.write("%s\n" % item)
output_pass.write("%s" % record[-1])

Using join is not recommended if you said the file was large - it will create the entire file content string in memory.

Upvotes: 14

ALSHARGI NEWYORK
ALSHARGI NEWYORK

Reputation: 157

it should work. use .replace("\n", "") for the last item in the list.

for item in items:
    print item[0], line[1].replace("\n", "")

Upvotes: 0

Bhargav Rao
Bhargav Rao

Reputation: 52071

You can join them first and then write as in

item = '\n'.join(record)
output_pass.write('%s' %item)

Note

If your list, i.e. record doesn't contain strings, then as Martinaeu has mentioned you will have to map it to a str that is, '\n'.join(map(str, record)) before you write to file. (In py2)

Upvotes: 1

martineau
martineau

Reputation: 123463

The following would write all but the last item in record with newlines very quickly and then the final one without it. It will do so without requiring much additional memory.

(For Python 3 use range instead of xrange)

item = iter(record)
for _ in xrange(len(record)-1):
    output_pass.write('%s\n' % next(item))

output_pass.write('%s' % next(item))

Upvotes: 1

yhoyo
yhoyo

Reputation: 207

do you try with some counter? like:

record = [str(x) for x in range(10)]
print record

import sys
output_pass=sys.stdout

counter = 0

while counter != (len(record))-1:
   output_pass.write("%s\n" % record[counter])
   counter += 1

Upvotes: 1

Tenzin
Tenzin

Reputation: 2505

Or you can make a definition to write a file away. 

def writeFile(value): 
    open('file.txt', 'a') as file 
    file.write(value)
    file.write('\n')

Then you call this definition to write data in the file. "value" will be in one line. 

writeFile('HelloWorld')

Upvotes: 0

Jonathan Vanasco
Jonathan Vanasco

Reputation: 15680

I don't think you should worry about a trailing \n. It's beneficial in a lot of scenarios (like if you want to add another line), and it's even recommended as part of the flake8 python source profiler.

@Amir's answer will work for avoiding writing the newline.

If you want to remove the last line, you can just write the entire file as-is, and then create a file object to use seek() + read() to test the last character, and then seek() and truncate() to remove it -- calling read after seek will require a second seek.

This is largely explained in this Q&A:

Upvotes: 0

Juan Lopes
Juan Lopes

Reputation: 10565

This requires constant additional memory:

for i, item in enumerate(record):
    if i>0: 
        output_pass.write('\n')
    output_pass.write('%s' %item)

Upvotes: 2

Related Questions