Last active
April 6, 2021 21:57
-
-
Save mark-kubacki/b42063a663db6b89a04aebd5af58baa5 to your computer and use it in GitHub Desktop.
Collect a set of repositories, each in its own subdir, into one mono-repo.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Merges a set of repositories, the “sources”, formerly hosted on “from” | |
# into subdirectories of one repository | |
# meant to be moved to a new repository hosting provider. | |
# | |
# git version: 2.24.0 | |
# Mark Kubacki, 2020-01-08 | |
if (( $# < 1 )); then | |
>&2 printf "Usage: $0 [repo | repo…]\n" | |
exit 2 | |
fi | |
set -euop pipefail | |
# Change this to your Gitlab, if need be. The 'slug' below will be %s. | |
: ${hosting:="https://source.developers.google.com/p/${USER}/r/%s"} | |
# This is the slug for the destination repository. | |
: ${D:="unrelated-containers"} | |
# I'm moving away from Github. | |
: ${leave:="[email protected]:${USER}/%s.git"} | |
declare -a sources=() | |
workdir="$(mktemp -d -t merge-repos.XXXXXX)" | |
cd "${workdir}" | |
trap "rm -rf '${workdir}'" EXIT | |
# Fetch all relevant repositories. | |
>&2 printf "==== git clone …\n" | |
for source; do | |
if [[ ! -d "${source}" ]]; then | |
printf -v from "${leave}" "${source}" | |
if ! git clone "${from}"; then | |
>&2 printf "ERR, skipping: %s ← %s\n" "${source}" "${from}" | |
continue | |
fi | |
fi | |
sources+=("${source}") | |
done | |
# Go through every source repository and move its contents one directory level down. | |
>&2 printf "==== push to subdirectories\n" | |
for source in "${sources[@]}"; do | |
cd "${source}" | |
FILTER_BRANCH_SQUELCH_WARNING=1 \ | |
git filter-branch -f --prune-empty --tree-filter ' | |
mkdir -p "'${source}'" | |
git ls-tree --name-only $GIT_COMMIT | xargs -I files mv files "'${source}'" | |
' | |
cd .. | |
done | |
# Pull everything into the super-repository. | |
>&2 printf "==== assemble the mono-repository ====\n" | |
printf -v destination "${hosting}" "${D}" | |
if git clone -o 'destination' "${destination}"; then | |
cd "${D}" | |
else | |
mkdir "${D}" | |
cd $_ | |
git init . | |
git commit --allow-empty -m "Create this mono-repository" | |
git remote add 'destination' "${destination}" | |
#gcloud source repos create ${D} | |
fi | |
for source in "${sources[@]}"; do | |
if [[ -d "${source}" ]]; then | |
>&2 printf "EXISTS, skipping: %s\n" "${source}" | |
continue | |
fi | |
git remote add -f "${source}" ../"${source}" | |
git merge -s ours --allow-unrelated-histories --no-commit "${source}/master" | |
# No need to work with another prefix: The files already are in distinct subdirectories. | |
git read-tree --prefix=""/ -u "${source}/master" | |
git commit -m "subtree-merge: ${source}" | |
done | |
git push destination master | |
cd .. | |
# fin | |
exit 0 |
If the source repository already contains a directory you want to move it entirely to, or below, — for example a cmd/
but everything shall go into cmd/indexer/…
—, do this:
cd strayrepo/
FILTER_BRANCH_SQUELCH_WARNING=1 \
git filter-branch -f --prune-empty --tree-filter '
mkdir -p "xmd/indexer"
git ls-tree --name-only $GIT_COMMIT | xargs -I files mv files "xmd/indexer"
'
FILTER_BRANCH_SQUELCH_WARNING=1 \
git filter-branch --tree-filter '
mv xmd cmd
' --force HEAD
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If you already have a super-repository or “target mono-repo”, you could do the following. For the sake of this example I'll call the destination directory
cmd/indexer
. Its git checkout is instrayrepo/
: