Last active
November 19, 2020 16:29
-
-
Save juanmc2005/4307ccf9f09f61f0fb99b4acaa618510 to your computer and use it in GitHub Desktop.
Project synchronization with remote server using rsync with exponential backoff. Just place the script at the root of the project and run.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Sync project with remote server using rsync with exponential backoff. | |
# Just copy it to the root of the project and make sure rsync is installed before running. | |
# | |
# Why I use this script: | |
# - sshfs doesn't play well with IntelliJ's file sync (IDE freezes quite often) | |
# - pushing silly commits for debugging or running experiments is not a good practice | |
# | |
# I built this script to reduce my network traffic and be more efficient in my workflow. | |
# Personally, I add a configuration for it in IntelliJ with the parameters I want for exponential backoff, | |
# then I just need to hit a button to send all updates to the server. | |
# | |
# Usage example: | |
# > sync.sh [user@host:]/path/to/remote/root/ [--dry] [--interval i] [--exp-rate r] [--max-retries m] | |
# | |
# AUTHOR: Juan Manuel Coria - https://juanmc2005.github.io/ | |
# Default values for constant arguments | |
dry=0 | |
dest="" | |
# Constants for exponential backoff | |
# Max retries is 5 | |
# Starts with interval of 1 second | |
# Exponential rate is 2 | |
let tries=0 interval=1 rate=2 maxr=5 | |
# Parse arguments | |
while [[ $# -gt 0 ]]; do | |
case "$1" in | |
--dry) dry=1 | |
;; | |
--interval) interval=$2 | |
shift | |
;; | |
--exp-rate) rate=$2 | |
shift | |
;; | |
--max-retries) maxr=$2 | |
shift | |
;; | |
--*) echo "Ignoring bad argument $1" | |
;; | |
*) dest=$1 | |
;; | |
esac | |
shift | |
done | |
# Destination path is mandatory | |
if [[ -z $dest ]]; then | |
echo "Please specify destination as [user@host:]/path/to/remote/root/" >&2 | |
exit 1 | |
fi | |
sync () { | |
if [[ $dry -eq 1 ]]; then | |
rsync -avz --dry-run --exclude "sync.sh" ./ $dest | |
else | |
rsync -avz --exclude "sync.sh" ./ $dest | |
fi | |
} | |
# First attempt | |
sync | |
# Continue trying if it keeps failing until success or max retries reached | |
while [[ $? -ne 0 && $tries -lt $maxr ]]; do | |
t=$(($interval*($rate**$tries))) | |
echo "Sync failed. Waiting ${t}s before retrying..." | |
sleep "$t"s | |
echo "Retrying now..." | |
let tries+=1 | |
sync | |
done | |
# Notify success/failure | |
if [[ $tries -ge $maxr ]]; then | |
if [[ $dry -eq 1 ]]; then | |
echo "Sync simulation (dry run) failed after ${maxr} retries" >&2 | |
exit 2 | |
else | |
echo "Sync failed after ${maxr} retries" >&2 | |
exit 3 | |
fi | |
elif [[ $dry -eq 0 ]]; then | |
echo "Sync successful" | |
fi |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment