Skip to content

Instantly share code, notes, and snippets.

@tkalfigo
Last active June 21, 2019 11:48
Show Gist options
  • Save tkalfigo/5695637 to your computer and use it in GitHub Desktop.
Save tkalfigo/5695637 to your computer and use it in GitHub Desktop.
Bash shell script for WAL archiving over ssh to remote host when doing replication with Postgresql
#!/bin/bash -x
PG_DATA_DIR="/home/postgresql/data";
DEST_HOST="example.com";
DEST_DIR="/home/remote_user/WAL_ARCHIVE";
SCP_USERNAME="remote_user";
# uses ssmtp to send email [for setup see: http://www.havetheknowhow.com/Configure-the-server/Install-ssmtp.html]
EMAIL_RECIPIENT="[email protected]";
EMAIL_MSG="To: [email protected]\nFrom: [email protected]\nSubject: Replication error\n\n";
#in case of error, how long to sleep before retrying
SLEEP_TIME=600;
#takes 1 arg: body of email
function sendEmail {
echo -e "$EMAIL_MSG\n\n$1" | /usr/sbin/ssmtp $EMAIL_RECIPIENT
}
# %p is passed from postgresql.conf's 'archive_command' parameter as $1 and is "pg_xlog/000000010000000000000014"
# i.e. relative to $PG_DATA_DIR
if [ "x$1" = "x" ];then
#echo "ERROR: Missing first parameter (WAL archive filename relative to PG_DATA)";
#echo "USAGE: $0 <WAL archive_filename relative to PG_DATA>";
#echo "USAGE EXAMPLE: $0 pg_xlog/000000010000000000000014";
#### SEND EMAIL AND EXIT
sendEmail "Missing 1st arg to call to \"$0\"" ;
sleep $SLEEP_TIME;
exit 1;
fi
if [ ! -f "$PG_DATA_DIR/$1" ];then
#echo "ERROR: Non-existent WAL file $PG_DATA_DIR/$1";
#echo "USAGE: $0 <WAL archive_filename relative to PG_DATA>";
#echo "USAGE EXAMPLE: $0 pg_xlog/000000010000000000000014";
#### SEND EMAIL AND EXIT
sendEmail "Non-existent WAL file \"$PG_DATA_DIR/$1\"";
sleep $SLEEP_TIME;
exit 2;
fi
#extract WAL filename from $1
WAL_FILENAME=`echo $1 | awk -F'/' '{print$2}'`;
#First check file doesn't already exist at destination; if it does => SEND EMAIL WITH ERROR and exit with failure
ssh $SCP_USERNAME@$DEST_HOST ls $DEST_DIR/$WAL_FILENAME;
if [ $? -eq 0 ];then
#echo "ERROR: WAL archive $WAL_ARCHIVE already exists at destination ($DEST_HOST:$DEST_DIR/$WAL_FILENAME)";
#echo "Local md5sum for $WAL_FILENAME: `md5sum $WAL_FILENAME";
#echo "Remot md5sum for $WAL_FILENAME: `ssh $SCP_USERNAME@$DEST_HOST md5sum $DEST_DIR/$WAL_FILENAME";
#### SEND EMAIL AND EXIT
sendEmail "WAL file \"$PG_DATA_DIR/$1\" already exists at destination $SCP_USERNAME@$DEST_HOST:$DEST_DIR/$WAL_FILENAME";
sleep $SLEEP_TIME;
exit 3;
else
# remote file doesn't exist; it's safe to scp to destination
/usr/bin/scp "$PG_DATA_DIR/$1" $SCP_USERNAME@$DEST_HOST:$DEST_DIR;
exit_code=$?;
if [ $exit_code -ne 0 ];then
sendEmail "Failed to scp WAL archive $PG_DATA_DIR/$1 to $SCP_USERNAME@$DEST_HOST:$DEST_DIR; exit code was $exit_code; sleeping and trying again";
#delay so user has time to see error-email and take action
sleep $SLEEP_TIME;
fi
#return with exit code of scp; provide it to Postgresql [anything non-zero means something went wrong] so it know whether to retry or to proceed
exit $exit_code;
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment