CODEWITHSUNDEEP
pythonfile-iooperating-system
tahoar
tahoar

Reputation: 1828

Does writing a file to disk with Python open().write() ensure the data is available to other processes?

One Python process writes status updates to a file for other processes to read. In some circumstances, the status updates happen repeatedly and quickly in a loop. The easiest and fastest approach is to use to open().write() in one line:

open(statusfile,'w').write(status)

An alternate approach with four lines that force the data to disk. This has a significant performance penalty:

f = open(self.statusfile,'w')

f.write(status)

os.fsync(f)

f.close()

I'm not trying to protect from an OS crash. So, does the approach force the data to the OS buffer so other processes read the newest status data when they open the file from disk? Or, do I need to use os.fsync()?

Upvotes: 8

Views: 17733

Answers (2)

Fred Foo
Fred Foo

Reputation: 363858

No, the first approach does not guarantee that the data is written out, since it is not guaranteed that the file will be flushed and closed once the handle is no longer referenced by its write member. This is likely the case with CPython, but not necessarily with other Python interpreters; it's an implementation detail of the Python garbage collector.

You should really use the second approach, except that os.fsync is not needed; just close the file and the data should be available to other processes.

Or, even better (Python >=2.5):

with open(self.statusfile, 'w') as f:
    f.write(status)

The with version is exception-safe: the file is closed even if write fails.

Upvotes: 13

martineau
martineau

Reputation: 123531

Since Python 2.2 it's been possible to subclass the language's built-in types. This means you could derive your own file type whose write() method returned self instead of nothing like the built-in version does. Doing so would make it possible to also chain a close() method call onto the end of your one-liner.

class ChainableFile(file):
    def __init__(self, *args, **kwargs):
        return file.__init__(self, *args, **kwargs)

    def write(self, str):
        file.write(self, str)
        return self

def OpenFile(filename, *args, **kwargs):
    return ChainableFile(filename, *args, **kwargs)

statusfile = 'statusfile.txt'
status = 'OK\n'

OpenFile(statusfile,'w').write(status).close()

Upvotes: 0

Related Questions