This is a quick and dirty script I came up with for a singular purpose: Back up my own repos on Gitlab to a directory on my home server. The idea is to preserve project files somewhere, independent of where they might live on the cloud.
Some will question the lack of any error handling and the cowardly use of shell commands, and rightly so. But this is a work in progress. Stay tuned for improvements!
My Gitlab API private token is currently set as an environment variable in my home directory (not the best security, for sure).
I only wrote this because I couldn't find anything better that I could grok (which is a progressively lowering bar).
See the docs for python-gitlab for more in-depth. FYI, this is an entirely different library from gitlab. Install python-gitlab with pip install python-gitlab (or sudo pip3 python-gitlab if you're on a Linux distro like Ubuntu and want it available globally). And yes, I think python-gitlab and gitlab's overlapping use of the same namespace is a huge problem.
Usage is like:
$ backup-gitlab.py --path /data/backup/projects
Note that this script will not automatically remove repositories that have been removed from Gitlab. That's something I prefer to do manually, after thinking about it over a cup of coffee.
#!/usr/bin/env python3
# Backup repositories of a Gitlab user.
import os
import gitlab
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--path", help="Backup path")
args = parser.parse_args()
if args.path:
backupdir = args.path
glab_token = os.environ.get('GLAB_TOKEN')
gl = gitlab.Gitlab('https://gitlab.com', private_token=glab_token)
projects = gl.projects.list(owned=True, all=True)
for project in projects:
ssh_url = project.ssh_url_to_repo
reponame = project.name
repodir = f'{backupdir}/{reponame}'
clone_cmd = f'git clone {ssh_url}'
pull_cmd = f'git pull'
print(reponame)
if (os.path.isdir(repodir)):
os.chdir(repodir)
os.system(pull_cmd)
else:
os.chdir(backupdir)
os.system(clone_cmd)Setting this up in cron is straightforward, if you understand how cron works. Here's how I do it:
GLAB_TOKEN="xxxxxxxxxxxxxxxxxxxxxxxxxxx"
...
0 22 * * * /usr/bin/python3 /home/myuser/bin/backup-gitlab.py --path /data1/backup/projects
...When running a python script in cron on Linux, keep in mind that it will normally not be aware of your personal environment. Because of that, be sure to use full paths to the python binary and the script. In this case I've also defined the environment variable for my token at the top of my crontab. I also could have put it in my .profile and sourced that file in the exec line, like this:
0 22 * * * . /home/myuser/.profile /usr/bin/python3 /home/myuser/bin/backup-gitlab.py --path /data1/backup/projectsNOTE: I've all of my important repos to github and so am not using this script anymore.
