Skip to content

Instantly share code, notes, and snippets.

@juanmc2005
Last active November 19, 2020 16:29
Show Gist options
  • Save juanmc2005/4307ccf9f09f61f0fb99b4acaa618510 to your computer and use it in GitHub Desktop.
Save juanmc2005/4307ccf9f09f61f0fb99b4acaa618510 to your computer and use it in GitHub Desktop.
Project synchronization with remote server using rsync with exponential backoff. Just place the script at the root of the project and run.
#!/bin/bash
# Sync project with remote server using rsync with exponential backoff.
# Just copy it to the root of the project and make sure rsync is installed before running.
#
# Why I use this script:
# - sshfs doesn't play well with IntelliJ's file sync (IDE freezes quite often)
# - pushing silly commits for debugging or running experiments is not a good practice
#
# I built this script to reduce my network traffic and be more efficient in my workflow.
# Personally, I add a configuration for it in IntelliJ with the parameters I want for exponential backoff,
# then I just need to hit a button to send all updates to the server.
#
# Usage example:
# > sync.sh [user@host:]/path/to/remote/root/ [--dry] [--interval i] [--exp-rate r] [--max-retries m]
#
# AUTHOR: Juan Manuel Coria - https://juanmc2005.github.io/
# Default values for constant arguments
dry=0
dest=""
# Constants for exponential backoff
# Max retries is 5
# Starts with interval of 1 second
# Exponential rate is 2
let tries=0 interval=1 rate=2 maxr=5
# Parse arguments
while [[ $# -gt 0 ]]; do
case "$1" in
--dry) dry=1
;;
--interval) interval=$2
shift
;;
--exp-rate) rate=$2
shift
;;
--max-retries) maxr=$2
shift
;;
--*) echo "Ignoring bad argument $1"
;;
*) dest=$1
;;
esac
shift
done
# Destination path is mandatory
if [[ -z $dest ]]; then
echo "Please specify destination as [user@host:]/path/to/remote/root/" >&2
exit 1
fi
sync () {
if [[ $dry -eq 1 ]]; then
rsync -avz --dry-run --exclude "sync.sh" ./ $dest
else
rsync -avz --exclude "sync.sh" ./ $dest
fi
}
# First attempt
sync
# Continue trying if it keeps failing until success or max retries reached
while [[ $? -ne 0 && $tries -lt $maxr ]]; do
t=$(($interval*($rate**$tries)))
echo "Sync failed. Waiting ${t}s before retrying..."
sleep "$t"s
echo "Retrying now..."
let tries+=1
sync
done
# Notify success/failure
if [[ $tries -ge $maxr ]]; then
if [[ $dry -eq 1 ]]; then
echo "Sync simulation (dry run) failed after ${maxr} retries" >&2
exit 2
else
echo "Sync failed after ${maxr} retries" >&2
exit 3
fi
elif [[ $dry -eq 0 ]]; then
echo "Sync successful"
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment