A while back I had my laptop stolen and while I had backups of all the important stuff my latest full-system backup was several months old since I have to dig out my backup harddrive from its alternate location every time I want to update the backup (I don't store my laptop and backup harddrive in the same building since theft/fire would mean loosing all copies).
So, being sick of imperfect backup solutions I decided to find a solution for online full-system backups that met the following criteria:
- 100% open source client
- Zero-knowledge encryption (server can't access data)
- Works on GNU/Linux
- Affordable (max $20 per month for ~1TB)
After some research I ended up using duplicity + backblaze. So far I have only run into one issue with this solution: duplicity doesn't support hardlinks. Luckily I don't use hardlinks very often.
duplicity is like rsync but with encryption so you can safely store your backups in an untrusted location.
backblaze provides low cost online storage.
As long as you don't need to restore your backups backblaze is extremely cheap :), and if you do end up needing your backups then backblaze is still affordable. duplicity recently added support for backblaze as a storage backend. duplicity allows you to restore single files so you won't be charged $20 every time you need something from you 2 TB laptop HD backup.
First, sign up for Backblaze B2 Cloud Storage on backblaze.com.
Create a bucket with some name that makes sense to you (I keep a bucket for each computer I want to back up).
You should add a credit card if you'll be using more data than the free amount.
Create an application key for your bucket and note down the keyID and applicationKey strings.
Install rsync:
sudo apt-get install rsync
Now install duplicity and the backblaze python package:
pip3 install duplicity
pip3 install b2
Save the following script to /usr/local/bin/backup
#!/bin/bash
KEY_ID="your_key_id"
APP_KEY="your_app_key"
BUCKET_NAME="your_bucket_name"
BACKUP="/" # what to back up
EXCLUDE=("/dev" "/media" "/mnt" "/proc" "/sys" "/tmp" "/run") # what to not back up
HARDDRIVES=("/dev/sda") # back up partition info for these harddrives
MBR_BACKUP="/opt/mbr_backup" # where to store partition info backup
if [[ $EUID -ne 0 ]]; then
echo "You must be root to read all system files for the backup"
exit 1
fi
# save mbr and partition information
mkdir -p $MBR_BACKUP
if [[ -d /etc/lvm/archive ]]; then
cp -a /etc/lvm/archive ./etc_lvm_archive # back up lvm partition info
fi
for HD in "${HARDDRIVES[@]}"; do
HD_NAME=`basename $HD`
dd if=${HD} of=${MBR_BACKUP}/${HD_NAME}.mbr bs=512 count=1 # save mbr
sfdisk -d $HD > ${HD_NAME}.sfdisk # save partition table (no lvm/crypto)
lsblk -f -b $HD > ${HD_NAME}.lsblk # save human readable partition tree for each hd
for PART in $(sfdisk -l ${HD} 2> /dev/null | grep "^/dev/" | awk '{ print $1 }'); do
# save blkid for each partition
PART_NAME=`basename $PART`
blkid -o value $PART | grep '^[0-9a-z]\{8\}-[0-9a-z]\{4\}-[0-9a-z]\{4\}-[0-9a-z]\{4\}-[0-9a-z]\{12\}' > "${MBR_BACKUP}/${PART_NAME}.blkid"
done
done
DUP_EXClUDE=""
for VAL in "${EXCLUDE[@]}"; do
DUP_EXCLUDE="$DUP_EXCLUDE --exclude $VAL"
done
echo "Running duplicity"
duplicity --progress --verbosity info backup $DUP_EXCLUDE $BACKUP "b2://${KEY_ID}:${APP_KEY}@${BUCKET_NAME}/"
Set permissions:
sudo chown root.root /usr/local/bin/backup
sudo chmod 700 /usr/local/bin/backup
Edit the file setting at least the first three variables. To get the account ID and application key you need to go to the Buckets page on backblaze and click the "Show Account ID and Application Key" link.
Note that this script creates the directory /opt/mbr_backup where it stores a copy of your master boot record, and partition information from each harddrive you specify in the the HARDDRIVES variable.
The first time you run this script it will take a long time since it's backing up your entire system. Every subsequent run will just create an incremental backup.
If you have any running databases then you should modify the script to save dumps of all databases to your hd before running the backup since you might otherwise end up with a backup of your database in an inconsistant state. You could also just shut down you databases before running the the script.
After running the backup script check that the amount of data in the Backblaze Bucket is about what you'd expect (using the Backblaze web interface).
Now before considering this backup successful you should attempt to recover at least one of the files from your backup on a different computer. It's all too easy to assume you have a good backup but forget to backup your encryption key or back up the wrong encryption key, so go to the next session and attempt to restore and ensure you do so on a different computer!
Also: Remember to back up your ~/.gnupg directory and remember that if you forget the passphrase you used for the backup then there is no way to recover!
Save the following script to the file /usr/local/bin/restore, ensure the ACCOUNT_ID, APP_KEY and BUCKET_NAME are the same as in the backup script form above.
#!/bin/bash
KEY_ID="your_key_id"
APP_KEY="your_app_key"
BUCKET_NAME="your_bucket_name"
if [ "$#" -ne "1" ]; then
echo "Attempting to restore everything to current directory"
duplicity --progress --verbosity info restore "b2://${KEY_ID}:${APP_KEY}@${BUCKET_NAME}/" "$TO_RESTORE" "."
else
echo "Attempting to restore $1 to current directory"
duplicity --progress --verbosity info restore --path-to-restore "$TO_RESTORE" "b2://${KEY_ID}:${APP_KEY}@${BUCKET_NAME}/" "$TO_RESTORE"
fi
Set permissions:
sudo chown root:root /usr/loca/bin/restore
sudo chmod 700 /usr/loca/bin/restore
To test restoring a small part of your previous backup run e.g:
mkdir /tmp/foo
cd /tmp/foo
restore opt/mbr_backup
Use a bootable usb with e.g. lubuntu.
ToDo how to restore single files
To restore your MBR and partition table from the backup files, for each harddrive do something like
sudo dd if=sda.mbr of=/dev/sda
sudo sfdisk /dev/sda < sda.sfdisk
Now you'll have to format each partition using your preferred filesystem. If you have encrypted partitions and/or use lvm then you'll have to manually re-establish that setup manually. If you don't remember how it was set up then the *.lsblk files should be helpful.
ToDo how to restore the rest of the filesystem.
Since your new partitions will have different block IDs you'll have to restore the old block IDs for each partition. Assuming the partitions are ext2, ext3 or ext4 partitions, for each partition do something like:
sudo tune2fs /dev/sda1 -U `cat sda1.blkid`
Now your system should be restored and ready to boot.
It would be great to have a script to automatically run this backup script, but it should probably have the following features:
- some algorithm to decide when enough files have changed to trigger a new backup
- bandwidth and cpu limiting so it runs without impacting work
- whitelist/blacklist of (wifi) networks where backups are allowed/disallowed
The last point is important since you might not want to have automatic backups when I'm tethering.
Thanks @jnny, this has now been fixed.