Skip to content

Instantly share code, notes, and snippets.

@seanorama
Last active August 7, 2019 13:45
Show Gist options
  • Save seanorama/88d7d21b22a0c4d495efdae35096c7b6 to your computer and use it in GitHub Desktop.
Save seanorama/88d7d21b22a0c4d495efdae35096c7b6 to your computer and use it in GitHub Desktop.
holland-backup-to-hdfs

Backup MySQL/MariaDB with holland backup

Including copying the backup to HDFS. Uses Kerberos by default, but you can strip that out.

  • If your environment is not using kerberos, remove the kinit/destroy commands.
  • Works with any holland module but this shows mysqldump.
  • Assumes credentials are in /root/.my.cnf.

Kinit to Smokeuser or whichever user you want the backup to run as in HDFS

keytab=/etc/security/keytabs/hdfs.headless.keytab
export KRB5CCNAME=/tmp/krb5cc_root_hollandbackup &&
  kinit -kt ${keytab} $(klist -kt ${keytab}|awk '{print $NF}'|tail -1)

Create directories in HDFS

## You must kinit 1st
hdfs dfs -mkdir -p /var/backups/mysql/data
hdfs dfs -chown -R ambari-qa /var
hdfs dfs -chmod -R 700 /var

Install Holland from EPEL

## install postgres or other modules if using that instead of mysql/mariadb
yum install holland holland-mysql holland-mysqldump

Create Holland default config

holland mc mysqldump --name default

Update the [holland] section with the text below in the created file

backups-to-keep = 7
after-backup-command = export KRB5CCNAME=/tmp/krb5cc_root_hollandbackup && kinit -kt /etc/security/keytabs/smokeuser.headless.keytab $(klist -kt /etc/security/keytabs/smokeuser.headless.keytab|awk '{print $NF}'|tail -1) && hadoop fs -copyFromLocal ${backupdir} /var/backups/mysql/data && kdestroy

Test backup

## You must kinit 1st
holland -v bk

Confirm it worked and landed in HDFS

hdfs dfs -ls -R /var

Add to cron

sudo tee /etc/cron.d/holland > /dev/null << EOF
30 1 * * * root /usr/sbin/holland -q bk
EOF
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment