File locking using fcntl.flock using Python

flock_example.py

"""
Example of using fcntl.flock for locking file. Some code inspired by filelock module.
"""

import os
import fcntl
import time


def acquire(lock_file):
    open_mode = os.O_RDWR | os.O_CREAT | os.O_TRUNC
    fd = os.open(lock_file, open_mode)

    pid = os.getpid()
    lock_file_fd = None
    
    timeout = 5.0
    start_time = current_time = time.time()
    while current_time < start_time + timeout:
        try:
            # The LOCK_EX means that only one process can hold the lock
            # The LOCK_NB means that the fcntl.flock() is not blocking
            # and we are able to implement termination of while loop,
            # when timeout is reached.
            # More information here:
            # https://docs.python.org/3/library/fcntl.html#fcntl.flock
            fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
        except (IOError, OSError):
            pass
        else:
            lock_file_fd = fd
            break
        print(f'  {pid} waiting for lock')
        time.sleep(1.0)
        current_time = time.time()
    if lock_file_fd is None:
        os.close(fd)
    return lock_file_fd


def release(lock_file_fd):
    # Do not remove the lockfile:
    #
    #   https://github.com/benediktschmitt/py-filelock/issues/31
    #   https://stackoverflow.com/questions/17708885/flock-removing-locked-file-without-race-condition
    fcntl.flock(lock_file_fd, fcntl.LOCK_UN)
    os.close(lock_file_fd)
    return None


def main():
    pid = os.getpid()
    print(f'{pid} is waiting for lock')
    
    fd = acquire('myfile.lock')

    if fd is None:
        print(f'ERROR: {pid} lock NOT acquired')
        return -1

    print(f"{pid} lock acquired...")
    time.sleep(2.0)
    release(fd)
    print(f"{pid} lock released")    


# You can run it using: python ./flock_example.py & python ./flock_example.py
if __name__ == '__main__':
    main()

    

Would it be more useful to do following in def acquire(lock_file):
First check if the file exists using os.path.exists(lock_file), if it does, open it in os.O_RDWR mode else open it in os.O_RDWR | os.O_CREAT mode and then try to get an exclusive lock using fcntl.flock(). One advantage of that is, process which does get the lock can write something useful (process id, hostname etc.) to the file and it will not get overwritten every time someone else tries to acquire the lock.
Of course, as existing file is NOT opened in O_TRUNC mode, if process having the lock dies and second process becomes the lock owner, it has to make sure to write at the beginning what it wants and then truncate rest of the contents. This is just to make sure, if number of bytes written by first process are more than the second, you get rid of the extra bytes.

It is just simple example. Of course you can extend it as you want. :-)

Thanks Jiri !! Am Fr., 17. Juni 2022 um 15:47 Uhr schrieb Jiri Hnidek < ***@***.***>:

…

***@***.**** commented on this gist. ------------------------------ It is just simple example. Of course you can extend it as you want. :-) — Reply to this email directly, view it on GitHub <https://gist.github.com/430d45c54311661b47fb45a3a7846537#gistcomment-4203882>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANFSZZINQW4UZQ6NMES25DVPRGFLANCNFSM5Y6W6MCA> . You are receiving this because you commented.Message ID: ***@***.***>

Super useful script, Thanks @jirihnidek

Very helpful example. Thanks.