CODEWITHSUNDEEP
pythonlinux
jldupont
jldupont

Reputation: 96854

Quick way to know if a file is open on Linux?

Is there a quick way (i.e. that minimizes time-to-answer) to find out if a file is open on Linux?

Let's say I have a process that writes a ton a files in a directory and another process which reads those files once they are finished writing, can the latter process know if a file is still being written to by the former process?

A Python based solution would be ideal, if possible.

Note: I understand I could be using a FIFO / Queue based solution but I am looking for something else.

Upvotes: 2

Views: 2033

Answers (11)

Maria Zverina
Maria Zverina

Reputation: 11173

If you know (or can find) the process ID of the writing process you can use psutil library. sudo pip install psutil to get the library. Docs are here: http://pythonhosted.org/psutil/

>>> import psutil
>>> import os
>>> p = psutil.Process(os.getpid())
>>> p.open_files()
[]
>>> f = open('foo.txt', 'w')
>>> p.open_files()
[openfile(path='/Users/mariaz/Downloads/foo.txt', fd=3)]

If you do not have access to the writing process, you'll need to run lsof as root and parse the output yourself.

Upvotes: 1

Jon Clements
Jon Clements

Reputation: 142256

I would use psutil ( https://github.com/giampaolo/psutil ) which also has the advantage of being cross-platform and provides many other useful system functions.

Upvotes: 1

pinepain
pinepain

Reputation: 12879

You can use fcntl module, afaik, it's has fcntl function that identical to C function, so something like fcntl(fd, F_GETFL) could be useful, but I'm not sure. Can you check is target file is blocked for writing by opening it in write mode?

Upvotes: 0

pwuertz
pwuertz

Reputation: 1325

This is a solution using inotify. You will get a notification for every file in the directory being closed after a writing operation.

import os
import pyinotify

def Monitor(path):
    class PClose(pyinotify.ProcessEvent):
        def process_IN_CLOSE(self, event):
            f = event.name and os.path.join(event.path, event.name) or event.path
            print 'close event: ' + f

    wm = pyinotify.WatchManager()
    notifier = pyinotify.Notifier(wm, PClose())
    wm.add_watch(path, pyinotify.IN_CLOSE_WRITE)

    try:
        while 1:
            notifier.process_events()
            if notifier.check_events():
                notifier.read_events()
    except KeyboardInterrupt:
        notifier.stop()
        return

if __name__ == '__main__':
    path = "."
    Monitor(path)

However, since you are the one being in control of the process writing the files I'd vote for a different solution involving some kind of communication between the processes.

Upvotes: 1

LSerni
LSerni

Reputation: 57453

If you know the PID of the writing process, in Linux you can simply query the /proc/{PID}/fd/ and see whether one of the links found there points to one of your files.

What you would do is, scan the directory, archiving the fact that fd 5 (say) points to /var/data/whatever/file1.log . Then store the file pointed to into an array.

At that point if a filename is in the array, the process has it in use.

So:

import os
# Here I use PID = 31824
path="/proc/%d/fd" % 31824
openfiles   = [ os.readlink("%s/%s" % (path, fname)) for fname in os.listdir(path) ]

if whatever in openfiles:
    # whatever is used by pid 31824.

Upvotes: 4

MattH
MattH

Reputation: 38265

You have a variety of options available:

  • Inotify is a feature that allows you to watch for file operations
  • Writing process renames files when finished writing
  • The program fuser will let you query whether a file is in use
  • Knowing the PID of the writer may let you query /proc/PID/fd for open file descriptors.

Upvotes: 2

tuxuday
tuxuday

Reputation: 3037

you can use lsof with subprocess

(output,error) = subprocess.Popen("lsof #absolute_file_path").communicate()

Upvotes: 1

hendry
hendry

Reputation: 10863

lsof | grep filename immediately comes to mind.

Upvotes: 2

Aleh
Aleh

Reputation: 700

If you can change the 'first' process logic, the easy solution would be to write data to a temp file and rename the file once all the data is written.

Upvotes: 1

sureshvv
sureshvv

Reputation: 4422

You can check the modification time of the file and see if it has not been modified for a period of time. Since a file can be opened in update mode and be modified any time, you cannot be 100% sure that it will never be modified.

Upvotes: 0

fork0
fork0

Reputation: 3459

You can of course use INOTIFY feature of Linux, but it is safer to avoid the situation: let the writing process create the files (say data.tmp) which the reading process will definitely ignore. When the writer finishes, it should just rename the file for the reader (into say .dat). The rename operation guarantees that there may be no misunderstandings.

Upvotes: 10

Related Questions