CODEWITHSUNDEEP
pythonfile-locking
David Davó
David Davó

Reputation: 688

How to filelock an entire directory?

I want to lock a directory and its contents, to allow only one process writing in it. To do that, I'm using the Portalocker library, and a file called .lock

from pathlib import Path
import os
import portalocker as pl

DIRECTORY = Path('./protected')
p_lock = DIRECTORY / '.lock'
with pl.Lock(p_lock, 'w', timeout=10) as lock:
    # Writing pid to lock
    print(os.getpid(), file=lock)
    lock.flush()

    # do things that write to some files
    # inside DIRECTORY

    # Remove lock file
    p_lock.unlink()

As long as all the processes execute the same code, no two process will be in the critical section. This wasn't causing any problems, but now I want to make a reader process.

from pathlib import Path
import os
import portalocker as pl

DIRECTORY = Path('./protected')
p_lock = DIRECTORY / '.lock'
with pl.Lock(p_lock, 'r', timeout=10):
    # do things

This throws a FileNotFoundError: [Errno 2] No such file or directory: './protected/.lock', because the file is removed when the writer finishes.

How can I make sure that either one process is writing, or multiple processes is reading to/from a directory?

Note: There is one writer process, and multiple readers (threads spawned from the same process, but not the same as the writer)

Upvotes: 0

Views: 927

Answers (1)

psarka
psarka

Reputation: 1842

This is an issue with how portalocker deals with files - it tries to open them, which works when opening non-existing file in 'w' mode, and fails in 'r' mode.

The solution in your case is to manually create the file (and never remove it, it's a bad idea).

Writer:

with pl.Lock(p_lock, 'w'):
    # do things
    # don't remove the p_lock file

Reader:

p_lock.touch(exist_ok=True)
with pl.Lock(p_lock, 'r', flags=pl.LockFlags.SHARED | pl.LockFlags.NON_BLOCKING):
    # do things
    # don't remove the p_lock file

(and don't bother writing PID into the file, unless it's for your own debugging purpuses)

Upvotes: 1

Related Questions