How can I delete a file after another process finishes with it?

Question

I have data held in memory that I want to open. I create a temporary file and fork xdg-open on ubuntu to open a file using the standard application. I want to delete the temporary file after the program closes. The problem is that xdg-open itself opens another process and quits immediately. Consequently, I have no idea what process opened the file. I have tried to get an exclusive lock with flock and also tried to unlink the file after I wait for 20 srcs (to make sure that the other process opens the file first), the calls succeed and the open program simply closes the file. I want to wait until the program is closed and then delete the file. Does anyone have any idea of how to accomplish this?

Answer 1

You can use either inotify (see man 7 inotify) or file leases (see man 2 fcntl, Leases section) to detect when other processes open and close the file (with inotify), or determine whether the file is open by other processes (file leases).

The core problem is that xdg-open is typically a script that examines the environment (and possibly the target file), then executes a binary (that in turn might examine the target file and execute another binary), and it is possible that one or more stages here fork and immediately exit, with the client process continuing the chain.

This means that the point in time when system() returns is basically irrelevant. The target file may or may not have been opened by the eventual application at that point; we just do not know, and have no way of knowing that.

One option would be to create a separate process group (session), and monitor the session, keeping the original file around for as long as the process group exists. However, that assumes that none of the scripts or binaries in the xdg-open chain create their own sessions. (I don't know if they do, there are so many different implementations -- each desktop environment uses their own binary, with xdg-open being the compatibility wrapper around them.)

In practice, that would mean replacing system() with your own implementation using fork(), setsid(), exec*(), and waitpid() and waitid(); the last one in a loop with a short sleep to detect when there are no more processes in the process group.

Another option is to execute the command, then (fork a child process to) wait for a specific period -- say, as long as an average user might tolerate to wait for the file to start loading; a few seconds, in other words --, then start checking whether the file is still in use. After the file is no longer used, it can be unlinked.

With inotify(), you'd assign the watch before executing the xdg-open command, and then monitor both opens and closes. Because it is possible that the xdg-open examines the target file to choose the application, you cannot assume the first close is a final close; you also need to wait for the specific period mentioned above, to make sure the application-open-chain has completed. Then, if there are as many closes as there were opens, the file can be unlinked. Otherwise, you'll wait for the remaining close(s), and unlink the file after the final one.

With file leases, the method is slightly simpler, but also more limited. You can only obtain file leases to normal files owned by the user itself. You can obtain a read lease only if the file is not open for writing by any process (including other descriptors by this same process). You can obtain a write lease only if the file is not open at all by any process (including other file descriptors by this same process). While you hold a lease, any other process opening the file (if you hold a write lease), or trying to modify it (if you hold a read or a write lease on it), will cause a SIGIO signal (by default, you can change it to a realtime signal) to be sent to the lease holder. It has up to /proc/sys/fs/lease-break-time seconds to downgrade or release the lease, until the kernel breaks it forcibly; during that time, the opener/file modifier will be blocked on the open()/truncate() call.

Before executing xdg-open, you can try obtaining a write lease on the file. If it succeeds, you know this is the only open file descriptor to it. After xdg-open is called, the lease will be broken when the file is opened (or examined by one of the binaries); you can simply release the lease before the call to avoid the hassle. After a suitable number of seconds has passed from when xdg-open was executed -- the amount of time a human would wait for the application to start opening the file --, you start periodically checking if the file is still open by some other process by trying to get a write lease on it. If the write lease succeeds, and enough time has passed from when you started the xdg-open, then you know that either the "human user" would have become too frustrated to wait any longer for the file to be opened, or the application has already closed the file, and therefore the file can be unlinked.

All of the above can be combined to get as paranoid as you wish, but personally, I believe the approach modeling human behaviour is most robust. I'd personally make the time limit (and write lease attempt interval) easily configurable, with something like 10 seconds and 1 second defaults, respectively.

Finally, if resource usage is a concern, then I recommend writing a separate helper binary to manage this for you. Basically, instead of running xdg-open [OPTIONS] FILENAME, you run /usr/lib/myapp/open DELAY INTERVAL [OPTIONS] FILENAME. The /usr/lib/myapp/open binary forks and immediately exits. The child process both executes xdg-open, and implements the procedure described above to wait until the file can be unlinked. Each of the /usr/lib/myapp/open binaries requires very little data (minimal resident set size) and resources (they mostly sleep), so even having a few dozen of them in memory won't put a significant drain even on an embedded Linux machine.

If there is interest, I could add an example C implementation of /usr/lib/myapp/open here. (Just let me know which of the three approaches is most interesting -- process group monitoring, inotify, or file leases.)

Answer 2

When xdg-open does not rely on the suffix, you can use

fd = open(filename, O_RDONLY|O_CLOEXEC);
unlink(filename);

system("xdg-open /proc/%u/fd/%u", getpid(), fd); /* pseudo-code! system is not printf() like! */

close(fd);