Skip to content

Instantly share code, notes, and snippets.

@plembo
Last active March 5, 2019 19:37
Show Gist options
  • Save plembo/f92a269fadc2ebffbe6cd9c5f50bd7b7 to your computer and use it in GitHub Desktop.
Save plembo/f92a269fadc2ebffbe6cd9c5f50bd7b7 to your computer and use it in GitHub Desktop.
Using rsync to mirror host folders

Using rsync to mirror host folders

If you don't need versioning or encryption here's a simple script that uses rsync to mirror select folders on a Linux host to a remote backup server.

#!/usr/bin/env bash
# Script to mirror host folders to backup

SHOST=$(`hostname|cut -f1 -d.`)
THOST='mybackuphost.example.com'
TROOT='/d1/backup';
EXCL='/usr/local/etc/excludes.conf'
SDIRS=('/etc' '/root' '/home' '/usr/local/bin' '/usr/local/etc' '/var/spool/cron/crontabs');
LOGFILE=/d1/logs/backup/mirror_host.log

TIMESTAMP=`date +%Y%m%d%H%M%S`
echo "${TIMESTAMP} Mirror ${SHOST} to ${THOST}" >${LOGFILE}

for SDIR in ${SDIRS[@]};
do
    rsync -avzR --exclude-from=${EXCL} --delete --log-file=${LOGFILE} ${SDIR} ${THOST}:${TROOT}/${SHOST}/

done

TIMESTAMP=`date +%Y%m%d%H%M%S`
echo "${TIMESTAMP} Mirroring completed" >>${LOGFILE}

The rsync options used here should be familiar, except for "-R", which preserves the full folder paths (without -R '/usr/local' would be mirrored as 'local', and '/var/spool/cron/crontabls' as 'crontabs'. It also will avoid unwanted deletions where a subfolder name shows up more than once, like '/home' and '/d1/home'.

Here is my excludes.conf file:

; Exclude file for rsync job (mostly targetting home directories)
tmp/
Downloads/
.cache/
.dbus/
.gvfs/
.mozilla/
.config/google-chrome/
Projects/
*/Cache/
.local/share/Trash/
lost+found/
.local/
.config/
node_modules/
.npm/
.nvm/
.gem/
.rbenv/
.vscode/
.eclipse/
.var/
*.sock
.virtualenvs/
.ssh/
.gnupg/
OneDrive/
GDrive/
.steam/
.gradle/
@plembo
Copy link
Author

plembo commented Mar 5, 2019

Updated this to reflect a re-ordering of the "--delete" and "--exclude-from" flags. My goals are to (a) filter out files that don't need backing up; and (b) delete any files in the target that have been removed from the source. The order "--exclude-from" ... "--delete" will accomplish both. I've also added the "--log-file" flag to capture info reported by the rsync as it runs, and expanded my exclusion list after some further experience.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment