CODEWITHSUNDEEP
python
jtcloud
jtcloud

Reputation: 559

Python write files process crashed before closing

If I have a process that is writing large number of lines on a file and the process got killed in the middle before the file handler is closed. Do I get the content of the lines written successfully?

Upvotes: 0

Views: 671

Answers (1)

tdelaney
tdelaney

Reputation: 77337

It depends on what you mean by "got killed" and "successfully". If its a python-raised exception then the file is closed (which flushes it to the operating system) because it exited a with clause or because the final reference to the object was removed and its __del__ method was called. This last bit isn't always guaranteed, which is why with clauses are usually used.

If the program was killed by an external unhandled exception or crashed because some c library has a bug, then the file isn't closed and whatever data was still in the local process cache is lost. Whatever had already been flushed to the operating system by clib will be there, but not the latest data. And it may be truncated on a memory block boundary. You can call flush often to reduce the amount of lost data. Or perhaps use a memory mapped file and direct io.

Now to the question of success. Depending on what you're doing, a partially written file may be the worst choice. In fact, I'd wager that's the case more often than not. I'm guilty of this as much as anybody but it makes sense to write to a temporary file and move it on success. Or alternately, deleting the file on failure. Parially written faulty data is a curse.

Upvotes: 1

Related Questions