Lockable script

lockable_script_boilerplate.sh

#!/bin/bash
# SPDX-License-Identifier: MIT
 
## Copyright (C) 2009 Przemyslaw Pawelczyk <[email protected]>
##
## This script is licensed under the terms of the MIT license.
## https://opensource.org/licenses/MIT
#
# Lockable script boilerplate
 
### HEADER ###
 
LOCKFILE="/var/lock/`basename $0`"
LOCKFD=99
 
# PRIVATE
_lock()             { flock -$1 $LOCKFD; }
_no_more_locking()  { _lock u; _lock xn && rm -f $LOCKFILE; }
_prepare_locking()  { eval "exec $LOCKFD>\"$LOCKFILE\""; trap _no_more_locking EXIT; }
 
# ON START
_prepare_locking
 
# PUBLIC
exlock_now()        { _lock xn; }  # obtain an exclusive lock immediately or fail
exlock()            { _lock x; }   # obtain an exclusive lock
shlock()            { _lock s; }   # obtain a shared lock
unlock()            { _lock u; }   # drop a lock
 
### BEGIN OF SCRIPT ###
 
# Simplest example is avoiding running multiple instances of script.
exlock_now || exit 1
 
# Remember! Lock file is removed when one of the scripts exits and it is
#           the only script holding the lock or lock is not acquired at all.

I don't know much about bash scripting, i have simple questions:

1. my script should go at the end, after exlock_now || exit 1?
2. do I have to call _no_more_locking explicitly at the end?

@luislobo:

1. Yes.
2. No, _no_more_locking is called automatically when script ends.

Sorry for late reply, but unfortunately gists lack any notification system...

Super helpful, thanks for this boilerplate. 🍹

@przemoc really useful snippet! But have you thought about getting it on the MIT license?

The GPL v2 prevents me form using it, as I would basically have to publish the script somewhere and that would be quite useless given that the script will have probably 2 more lines :)

Let me know what you think.

@PiotrCzapla
There won't be a problem. I'll improve this boilerplate a bit by incorporating some of my comments you can find in http://stackoverflow.com/a/1985512/241521 and relicense it to MIT License (which I'm already using in my more recent creations). For the time being please use simply relicensed version I've sent to your github e-mail.

@przemoc any news about the improved version?

Yes, would really like the MIT licensed version. "derivative work" can be a tricky with bash scripts.

As suggestion would be making it a sourceable library also that could be used across multiple scripts. Perhaps taking the lockname rather than using basename.

commit 6571db3000d4060e4168c3406f370a02a32b5045 (2017-01-10 23:54:05 +0100):

lockable_script_boilerplate.sh: Relicense to MIT.

I wanted to do it long time ago, along with some updates, but it's so
low on my todo list, that only comment on stackoverflow changed it:

    http://stackoverflow.com/a/1985512/241521

because I pasted the code there.

Sorry, no code updates yet.

commit 74a5939bc7729bb0b46de47ed0ef6f551c29a8f6
Author: Przemyslaw Pawelczyk <[email protected]>
Date:   2018-02-06 14:12:23 +0100

    Add SPDX License Identifier.

    The Software Package Data Exchange (SPDX) is a good initiative, it has
    matured over time and deserves accelerated adoption in open-source.

    https://spdx.org/learn
    https://spdx.org/using-spdx
    https://spdx.org/license-list

Hi,

I was evaluating this solution, the code works in the common case, but IMHO it has some formal issues.

AFAIU synchronization via flock is performed on the actual file on the filesystem (or in memory), not on the file name nor on the file descriptor, these latter are just convenience mechanisms to access the file end hence the lock.

Some example to verify that:

• using different file names for LOCKFILE would still work if they were (hard) linked to the same file, e.g.:

#this is just an example to show the point, cleanup is ignored.
ln "/tmp/EXISTING_FILE" "/tmp/lock.$$"
LOCKFILE="/tmp/lock.$$"

• using different file descriptors across invocations would still work (this is rather obvious considering that file descriptors are unique only in the same process) as long as they were bound to the same file, e.g.:

LOCKFD=$(( (RANDOM % 100) + 1))

Conversely, if the same file name refers to different files at two different points in time, locking would not work as desired.

For example:

exec 9>/tmp/filename

# removing the filename, but the file is still accessible via fd 9
rm /tmp/filename

# same name, new, different file (also different fd but this is unrelated)
exec 10>/tmp/filename

echo "Hello" >> /proc/self/fd/9
echo "World" >> /proc/self/fd/10

# The two files are different
cat /proc/self/fd/9
cat /proc/self/fd/10

the two file descriptors refer to different files in memory, even if they were opened with the same file name.

Considering that, I think the proposed solution leaves a window between _prepare_locking() and _lock() which can be problematic.

If for some reason $LOCKFILE gets deleted by some other entity when the first invocation is between _prepare_locking() and _lock(), a second invocation would fail to synchronize, because this second _prepare_locking() would create a new file (even when using the same name), so it would execute regardless of the first invocation.

To reduce the window to the minimum _lock and _prepare_lock could be merged into one function, there will be a little extra work if the script locks and unlocks multiple times but it will be more robust.

Finally, about _no_more_locking() consider adding a comment to explain that it is like that to support exlock or shlock; in case only exlock_now() is required the function would not be needed and just a trap "rm -f \"$LOCKFILE\";" would suffice since the lock will be released automatically when the process exits and the file is closed.

FWIW I ended up using the mkdir approach (http://wiki.bash-hackers.org/howto/mutex) in my scripts which looks simpler and more robust.

Sorry for the long post.

Ciao,
Antonio

An ugly test - if I kill -9 pid-of-my-script, then it doesn't have a chance to remove the lock.

An ugly test - if I kill -9 pid-of-my-script, then it doesn't have a chance to remove the lock.

This will always be true. kill -9 is "non-catchable, non-ignorable kill", the OS won't let you do anything when you get one, it's the equivalent of pulling taking an axe to the power supply. It's why you should escalate to kill -9 when trying to fix something, not start there.

This is beautiful. Thank you.

This is very nice.
If you don't mind the lockfile staying around, this is very simple and easy in bash and probably fine in sh (not tested):

exec 9>>mylockfile

if ! flock --exclusive --nonblock 9; then
    echo "$(date -u +"%FT%T.%3NZ") $$: scriptname will wait for lock"
    flock --exclusive 9
    echo "$(date -u +"%FT%T.%3NZ") $$: scriptname finished waiting for lock"
fi

... this code is protected against concurrency ...

It's easy enough to adjust that to exit if the script is already running:

exec 9>>mylockfile

if ! flock --exclusive --nonblock 9; then
    echo "scriptname already running, exiting"
    exit 1
fi

... this code is protected against concurrency ...

The lock is automatically released once the file descriptor is closed (which is inherited by subshells or things you exec from the shellscript, so all that needs to close).
Thus the rest of the script is protected against concurrent execution.
There is no issues with power loss and the lockfile staying around either.