Reputation: 96854
Is there a quick way (i.e. that minimizes time-to-answer) to find out if a file is open on Linux?
Let's say I have a process that writes a ton a files in a directory and another process which reads those files once they are finished writing, can the latter process know if a file is still being written to by the former process?
A Python based solution would be ideal, if possible.
Note: I understand I could be using a FIFO / Queue based solution but I am looking for something else.
Upvotes: 2
Views: 2033
Reputation: 11173
If you know (or can find) the process ID of the writing process you can use psutil library. sudo pip install psutil
to get the library. Docs are here: http://pythonhosted.org/psutil/
>>> import psutil
>>> import os
>>> p = psutil.Process(os.getpid())
>>> p.open_files()
[]
>>> f = open('foo.txt', 'w')
>>> p.open_files()
[openfile(path='/Users/mariaz/Downloads/foo.txt', fd=3)]
If you do not have access to the writing process, you'll need to run lsof as root and parse the output yourself.
Upvotes: 1
Reputation: 142256
I would use psutil ( https://github.com/giampaolo/psutil ) which also has the advantage of being cross-platform and provides many other useful system functions.
Upvotes: 1
Reputation: 12879
You can use fcntl module, afaik, it's has fcntl function that identical to C function, so something like fcntl(fd, F_GETFL)
could be useful, but I'm not sure. Can you check is target file is blocked for writing by opening it in write mode?
Upvotes: 0
Reputation: 1325
This is a solution using inotify. You will get a notification for every file in the directory being closed after a writing operation.
import os
import pyinotify
def Monitor(path):
class PClose(pyinotify.ProcessEvent):
def process_IN_CLOSE(self, event):
f = event.name and os.path.join(event.path, event.name) or event.path
print 'close event: ' + f
wm = pyinotify.WatchManager()
notifier = pyinotify.Notifier(wm, PClose())
wm.add_watch(path, pyinotify.IN_CLOSE_WRITE)
try:
while 1:
notifier.process_events()
if notifier.check_events():
notifier.read_events()
except KeyboardInterrupt:
notifier.stop()
return
if __name__ == '__main__':
path = "."
Monitor(path)
However, since you are the one being in control of the process writing the files I'd vote for a different solution involving some kind of communication between the processes.
Upvotes: 1
Reputation: 57453
If you know the PID of the writing process, in Linux you can simply query the /proc/{PID}/fd/ and see whether one of the links found there points to one of your files.
What you would do is, scan the directory, archiving the fact that fd 5 (say) points to /var/data/whatever/file1.log . Then store the file pointed to into an array.
At that point if a filename is in the array, the process has it in use.
So:
import os
# Here I use PID = 31824
path="/proc/%d/fd" % 31824
openfiles = [ os.readlink("%s/%s" % (path, fname)) for fname in os.listdir(path) ]
if whatever in openfiles:
# whatever is used by pid 31824.
Upvotes: 4
Reputation: 38265
You have a variety of options available:
/proc/PID/fd
for open file descriptors.Upvotes: 2
Reputation: 3037
you can use lsof with subprocess
(output,error) = subprocess.Popen("lsof #absolute_file_path").communicate()
Upvotes: 1
Reputation: 700
If you can change the 'first' process logic, the easy solution would be to write data to a temp file and rename the file once all the data is written.
Upvotes: 1
Reputation: 4422
You can check the modification time of the file and see if it has not been modified for a period of time. Since a file can be opened in update mode and be modified any time, you cannot be 100% sure that it will never be modified.
Upvotes: 0
Reputation: 3459
You can of course use INOTIFY feature of Linux, but it is safer to avoid the situation: let the writing process create the files (say data.tmp) which the reading process will definitely ignore. When the writer finishes, it should just rename the file for the reader (into say .dat). The rename operation guarantees that there may be no misunderstandings.
Upvotes: 10