CODEWITHSUNDEEP
python
Brent O'Connor
Brent O'Connor

Reputation: 5881

Why does tempfile.NamedTemporaryFile() truncate my data?

Here is a test I created to recreate a problem I was having when I used tempfile.NamedTemporaryFile(). The problem is that when I use tempfile the data in my CSV is truncated off the end of the file.

When you run this test script, temp2.csv will get truncated and temp1.csv will be the same size as the original CSV.

I'm using Python 2.7.1.

You can download the sample CSV from http://explore.data.gov/Energy-and-Utilities/Residential-Energy-Consumption-Survey-RECS-Files-A/eypy-jxs2

#!/usr/bin/env python

import tempfile
import shutil


def main():
    f = open('RECS05alldata.csv')
    data = f.read()
    f.close()

    f = open('temp1.csv', 'w+b')
    f.write(data)
    f.close()

    temp = tempfile.NamedTemporaryFile()
    temp.write(data)
    shutil.copy(temp.name, 'temp2.csv')
    temp.close()

if __name__ == '__main__':
    main()

Upvotes: 4

Views: 6044

Answers (3)

01100110
01100110

Reputation: 2344

Add temp.flush() after temp.write(data).

Upvotes: 12

Sam Mussmann
Sam Mussmann

Reputation: 5993

I think your problem is that Python has not flushed the entire file to disk when you call shutil.copy.

Change

temp = tempfile.NamedTemporaryFile()
temp.write(data)
shutil.copy(temp.name, 'temp2.csv')
temp.close()

to

temp = tempfile.NamedTemporaryFile()
temp.write(data)
temp.close()
shutil.copy(temp.name, 'temp2.csv')

Upvotes: 0

Mark Ransom
Mark Ransom

Reputation: 308206

You copy the file before you close it. Files are buffered, which means that some of it will remain in the buffer while it is waiting to be written to the file. The close will write out all remaining data from the buffer to the file as part of the closing of the file.

This has nothing to do with NamedTemporaryFile.

Upvotes: 3

Related Questions