CODEWITHSUNDEEP
pythonwindowsfilelocking
ewerybody
ewerybody

Reputation: 1590

How to test file locking in Python

So I want to write some files that might be locked/blocked for write/delete by other processes and like to test that upfront.

As I understand: os.access(path, os.W_OK) only looks for the permissions and will return true although a file cannot currently be written. So I have this little function:

def write_test(path):
    try:
        fobj = open(path, 'a')
        fobj.close()
        return True
    except IOError:
    return False

It actually works pretty well, when I try it with a file that I manually open with a Program. But as a wannabe-good-developer I want to put it in a test to automatically see if it works as expected.

Thing is: If I just open(path, 'a') the file I can still open() it again no problem! Even from another Python instance. Although Explorer will actually tell me that the file is currently open in Python!

I looked up other posts here & there about locking. Most are suggesting to install a package. You migth understand that I don't wanna do that to test a handful lines of code. So I dug up the packages to see the actual spot where the locking is eventually done...

fcntl? I don't have that. win32con? Don't have it either... Now in filelock there is this:

self.fd = os.open(self.lockfile, os.O_CREAT|os.O_EXCL|os.O_RDWR)

When I do that on a file it moans that the file exists!! Ehhm ... yea! That's the idea! But even when I do it on a non-existing path. I can still open(path, 'a') it! Even from another python instance...

I'm beginning to think that I fail to understand something very basic here. Am I looking for the wrong thing? Can someone point me into the right direction? Thanks!

Upvotes: 2

Views: 8070

Answers (2)

ewerybody
ewerybody

Reputation: 1590

Alright! Thanks to those guys I actually have something now! So this is my function:

def lock_test(path):
    """
    Checks if a file can, aside from it's permissions, be changed right now (True)
    or is already locked by another process (False).

    :param str path: file to be checked
    :rtype: bool
    """
    import msvcrt
    try:
        fd = os.open(path, os.O_APPEND | os.O_EXCL | os.O_RDWR)
    except OSError:
        return False

    try:
        msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
        msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
        os.close(fd)
        return True
    except (OSError, IOError):
        os.close(fd)
        return False

And the unittest could look something like this:

class Test(unittest.TestCase):

    def test_lock_test(self):
        testfile = 'some_test_name4142351345.xyz'
        testcontent = 'some random blaaa'
        with open(testfile, 'w') as fob:
            fob.write(testcontent)

        # test successful locking and unlocking
        self.assertTrue(lock_test(testfile))
        os.remove(testfile)
        self.assertFalse(os.path.exists(testfile))

        # make file again, lock and test False locking
        with open(testfile, 'w') as fob:
            fob.write(testcontent)
        fd = os.open(testfile, os.O_APPEND | os.O_RDWR)

        msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
        self.assertFalse(lock_test(testfile))

        msvcrt.locking(fd, msvcrt.LK_UNLCK, 1)
        self.assertTrue(lock_test(testfile))
        os.close(fd)

        with open(testfile) as fob:
            content = fob.read()
            self.assertTrue(content == testcontent)
        os.remove(testfile)

Works. Downsides are:

  • It's kind of testing itself with itself
  • so the initial OSError catch is not even tested, only locking again with msvcrt

But I dunno how to make it better now.

Upvotes: 0

Jose Raul Barreras
Jose Raul Barreras

Reputation: 859

You are trying to implement the file locking problem using just the system call open(). The Unix-like systems uses by default advisory file locking. This means that cooperating processes may use locks to coordinate access to a file among themselves, but uncooperative processes are also free to ignore locks and access the file in any way they choose. In other words, file locks lock out other file lockers only, not I/O. See Wikipedia.

As stated in system call open() reference the solution for performing atomic file locking using a lockfile is to create a unique file on the same file system (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful.

That is why in filelock they also use the function fcntl.flock() and puts all that stuff in a module as it should be.

Upvotes: 1

Related Questions