karlvr

Migrate from hosted GitLab to GitHub

In order to migrate from hosted GitLab to GitHub, I decided to use command-line git to mirror each repository from within the GitLab storage on our server.

The script uses the GitHub API to create a new private repository, and then uses the git command-line tool to mirror the repository.

The script keeps track of which repositories have been migrated by creating hidden .migrated-github-<repository> files. If you run the script again, these repositories will be skipped. This means you can re-run the script to recover from errors, such as a repository containing a file larger than 100MB (see https://rtyley.github.io/bfg-repo-cleaner/ for a solution to that).

If you choose to remove those files, you can still run the script multiple times as it mirrors the repository to GitHub. This approach could be used to pick up any changes made in GitLab before the final changeover to GitHub. But see the next paragraph...

karlvr

#!/bin/bash -eu
# Update the git remotes on git working copies contained in the given parent folder.
#
# usage: bulk-update-git-remotes.sh <path containing git repo(s)> <remote search> <remote replace>
#
# e.g. bulk-update-git-remotes.sh ~/git github.com/foo gitlab.com/bar

dryrun=0

while getopts ":n" opt; do

karlvr

function search(ob, needle, path = [], seen = new Map()) {
  if (seen.has(ob)) {
    return
  }
  seen.set(ob, true)

  for (const key of Object.keys(ob)) {
    if (typeof ob[key] === 'string') {
      if (ob[key].indexOf(needle) !== -1) {
        const pathDesc = [...path, key].map(pathKey => Number(pathKey) == pathKey ? `[${pathKey}]` : `.${pathKey}`).join('').substring(1)

karlvr

I have a heterogenous set of hosts and a mix of different roles that I want to apply to each host. Using groups would mean creating a group for nearly every role, which felt like overkill.

This combination of a playbook and two task scripts runs the roles specified in the host's required_roles variable, in order. It supports a tag named after the role, to run the specific role, and a tag role-partial to activate the role but to require other tags to activate specific tasks in the role (helpful when debugging roles).

karlvr

body {
	background-image: url('data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxNiIgaGVpZ2h0PSIxNyIgdmlld0JveD0iMCAwIDE2IDE3Ij4KICA8cGF0aCBmaWxsPSIjRkZGRkZGIiBmaWxsLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0xNiw4IEwxNiw0LjAwNjg0NTQ3IEMxNiwzLjQ0OTk0ODc2IDE1LjU1MTg5NDUsMyAxNC45OTkxMjgzLDMgTDEzLDMgTDEzLDUuMDAwMTkyNTEgQzEzLDUuNTUyMzcwOTQgMTIuNTU2MTM1Miw2IDEyLDYgQzExLjQ0NzcxNTMsNiAxMSw1LjU1NjI4MzM1IDExLDUuMDAwMTkyNTEgTDExLDMgTDUsMyBMNSw1LjAwMDE5MjUxIEM1LDUuNTUyMzcwOTQgNC41NTYxMzUxOCw2IDQsNiBDMy40NDc3MTUyNSw2IDMsNS41NTYyODMzNSAzLDUuMDAwMTkyNTEgTDMsMyBMMS4wMDA4NzE2NiwzIEMwLjQ0NDYzMDg2MSwzIDAsMy40NTA3ODAwNyAwLDQuMDA2ODQ1NDcgTDAsOCBMMTYsOCBaIE0xNiw5IEwxNiwxNS45OTMxNTQ1IEMxNiwxNi41NDkyMTk5IDE1LjU1NTM2OTEsMTcgMTQuOTk5MTI4MywxNyBMMS4wMDA4NzE2NiwxNyBDMC40NDgxMDU1MDUsMTcgMCwxNi41NTAwNTEyIDAsMTUuOTkzMTU0NSBMMCw5IEwxNiw5IFogTTQsMiBDMy40NDc3MTUyNSwyIDMsMS41NTIyODQ3NSAzLDEgQzMsMC40NDc3MTUyNSAzLjQ0NzcxNTI1LDAgNCwwIEM0LjU1MjI4NDc1LDAgNSwwLjQ0NzcxNTI1IDUsMSBDNSwxLjU1MjI4NDc1IDQuNTUyMjg0NzUsMiA0LDIgWiBNMTI

karlvr

#!/bin/bash -eu
#
# Delete tags from a git repository that reference commits that aren't part of the named branch
#
# This is useful if you've split a branch out of a repository and want to remove all of the commits
# that referenced activity in other branches.

usage() {
	echo "usage: $0 [-d] [-p <remote>] <branch>" >&1
	echo "  -d dry run" >&1

karlvr

#!/bin/bash -eu
# Download and update OpenSceneryX from https://www.opensceneryx.com

target="${1:-}"
if [ -z "$target" ]; then
	echo "usage: $0 <target dir>" >&2
	exit 1
fi

if [ ! -d "$target" ]; then