How to mirror a repository using GitLab CI

0_Mirror a repository using GitLab CI.md

Mirroring a repository using GitLab CI

inspired by https://gitlab.com/freifunkks/mirror-scripts

This can be used, for example, to clone a repository living on github over to GitLab in order to run CI

Setup

• Setup separate repository with mirror scripts. We'll call it "mirror-scripts"

• push the two files below.

  • in the .gitlab-ci.yml replace MY_REPO with your repo name
  • replace MY_USERNAME with your username
  • you can add multiple repos to mirror by copying the mirror job

• generate an SSH key.

  • Put the private key into a variable called SSHKEY_PROJECT1 on the mirror-scripts repo.
  • put the public key as deploy key into the source repo
  • put the public key as a write-enabled deploy key into the target repo

• in gitlab settings for the mirror-scripts repo:

  • enable pipelines
  • add a pipeline trigger hook. Note the token.
  • Also note the numeric project id. You can find it in the documentation snippet below the pipeline trigger settings.

• in the settings of the source repo:

  • add a webhook with this url: https://gitlab.com/api/v4/projects/PROJECT_ID/ref/master/trigger/pipeline?token=TOKEN
  • replace PROJECT_ID and TOKEN with the values from above
  • set the webhook to run on push

for more repos (ex PROJECT2):

• copy the block mirror-PROJECT1 in gitlab-ci.yml and replace PROJECT1 by PROJECT2
• generate a new SSH key
  • configure it for the source and target repo
  • put it into a variable called SSHKEY_PROJECT2

done!

1_.gitlab-ci.yml

before_script:
- apk add git openssh-client
- eval $(ssh-agent -s)
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

mirror-PROJECT1:
  stage: deploy
  script:
  - echo "$SSHKEY_PROJECT1" | ssh-add -
  - ./mirror.sh "PROJECT1" "[email protected]:MYSELF/PROJECT1" "[email protected]:MYSELF/PROJECT1.git"
  only:
    variables:
      - $REPO == "PROJECT1"
  cache:
    key: PROJECT1-cache-always
    paths:
    - PROJECT1.git

# -- more projects: --

mirror-PROJECT2:
  stage: deploy
  script:
  - echo "$SSHKEY_PROJECT2" | ssh-add -
  - ./mirror.sh "PROJECT2" "[email protected]:MYSELF/PROJECT2" "[email protected]:MYSELF/PROJECT2.git"
  only:
    variables:
      - $REPO == "PROJECT2"
  cache:
    key: PROJECT2-cache-always
    paths:
    - PROJECT2.git

2_mirror.sh

#!/bin/sh

set -euxo pipefail

NAME=$1
SOURCE_URL=$2
TARGET_URL=$3

if [[ -d "$NAME.git" ]]; then
    cd "$NAME.git"
    git fetch --prune --tags --prune-tags "$SOURCE_URL" "+refs/*:refs/*"
else
    git clone --bare "$SOURCE_URL"
    cd "$NAME.git"
fi

git push --mirror "$TARGET_URL"