Shell Locking Recipe for `producer > >(consumer)`

lockrecipe.sh

#!/bin/bash
#
# Locking example for on how to synchronize background processes.
# Here is what this is for:
# producer() { a=2; echo hello world; return 1; }
# consumer() { a=3; sleep 1; cat; return 2; }
# a=1; producer | consumer   ; echo a=$a # gives hello world; a=1
# a=1; producer > >(consumer); echo a=$a # gives a=2; hello world
# a=1; consumer < <(producer); echo a=$a # gives hello world; a=3
# As seen the a=2 output case is a bit asynchronous/race conditioned.
# This recipe here fixes that.

producer()
{
echo Hello
echo World
echo BYE
}

consumer()
{
echo ".."
while read -r a; do echo "--$a--"; sleep 1; echo "== $a =="; done
echo "##"
}

# This is a simple locking example.
# This recipe can have a race in case the machine is very busy
# so the `sleep 2` is faster than the inner `flock 6`
# (you can simulate this by writing `(sleep 3;exec` instead of `(exec`)
simple()
{
local lock="$(mktemp)"
{
rm -f "$lock"
producer > >(exec 6>/proc/$$/fd/6; flock 6; consumer)
echo AAAAA; sleep 2; echo WAIT; flock 6; echo LOCKED
} 6>>"$lock"
}

# Same as simple, but without race
# You need 2 locks for synchronization purpose
advanced()
{
{
local lock1="$(mktemp)" lock2="$(mktemp)"
{
rm -f "$lock1" "$lock2"
flock 7
producer > >(exec 6>/proc/$$/fd/6; flock 6; flock -u 7; consumer)
flock 7 7>/proc/$$/fd/7
flock 6 6>/proc/$$/fd/6
} 6>>"$lock1" 7>>"$lock2"
}

echo "simple:"
simple
echo "advanced:"
advanced

If you wonder what the /proc/$$/fd/7 is for:

The lock is shared over file descriptors across fork() and dup(). This means, a sequence like flock 6; flock 6 does not do two independent locks, it just repeats the first lock.

To be able to wait for a lock, you need to independently lock it again, so you need an independent file descriptor, and such an independent can be acquired only via a call to open().

However, we want to remove the tmpfiles (created by mktmp) as quickly as possible, such that you can interrupt the script anywhere with no need to cleanup later. But there is no file name anymore after the files were removed.

Luckily you can open() file descriptors of a running process ($$) via /proc/PID/fd/FD. Even that this two flock look a bit funny, they are correct. You do not need to keep the lock, they are only used to do the waiting.

Note that you could write it this way, too: flock 1 >/proc/$$/fd/7, but I stick to flock 7 7>/proc/$$/fd/7 because this makes it easier to grep for things like hidden flock 7 in bigger sources.