Debugging Gitlab CI without installing gitlab-runner

giltab-runner-local-docker.md

Debugging Gitlab CI jobs locally

You can find a bunch of articles about debugging Gitlab CI jobs locally. My top results from google:

• [SmallTips] Running GitLab CI Runner Locally | AkitaOnRails.com
• How to: Debug GitLab CI Builds Locally | Substrakt
• Debugging Gitlab CI pipelines locally – Campfire Code – Medium

But in all that articles the described way is to install Docker and gitlab-runner executable, run gitlab-runner exec docker JOB_NAME_FROM_GITLAB_CI_YML after that in your repo.

I don't like this approach because I want a simpler way to run appropriate gitlab-runner executable and don't want to have this executable in my system. Why do it if we can run it inside docker? And gitlab also provides appropriate images on dockerhub: gitlab/gitlab-runner.

Requirements:

• Docker

Quickstart

Run inside the root directory of your repo:

docker run --rm -ti \
            -v `pwd`:/`pwd`:ro \
            -v /var/run/docker.sock:/var/run/docker.sock \
            -w `pwd` \
            gitlab/gitlab-runner:alpine \
            exec docker JOB_NAME_FROM_GITLAB_CI_YML \
            --docker-volumes /var/run/docker.sock:/var/run/docker.sock

What it actually does

Image gitlab/gitlab-runner:alpine has custom entrypoint that actually is wrapper around gitlab-runner command. Look to the official repo of gitlab-runner for more information.

Also we need to allow runner inside container to connect to our local docker socket. We just map it as volume inside -v /var/run/docker.sock:/var/run/docker.sock.

We need to map our current directory again as volume with parameter -v pwd:/pwd:ro in read-only (just in case if something will go wrong). Directory inside the container should be the same as at our local machine because of gitlab-runner will create another container, try to map another volume inside the newly created container, but docker is run on our host machine and it can map only our local directories.

To run a job from .gitlab-ci.yml we need to pass the parameters exec docker JOB_NAME_FROM_GITLAB_CI_YML.

• exec is for executing job locally
• docker is for choosing Docker executor

Optionally if you want to run jobs that utilize docker you need an additional step that is not described in documentation. Otherwise jobs can fail with error: docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?. In current version gitlab-runner exec omit reading configuration from config.toml (here is mention about it) and in default configuration it will not pass /var/run/docker.sock inside container where job is executed. Only one way to do it is pass another argument --docker-volumes /var/run/docker.sock:/var/run/docker.sock to gotlab-runner executable.

gitlab-runner is very limited in the local environment without Gitlab orchestration. Also, it does not support all features Gitlab CI provides. For example we are not able to run CI jobs if from files in included option. It's usable for simple flows without artifacts, caches etc. So, I'm afraid that submodule configuration is not possible at all.

I hope that Gitlab will implement local debugging properly: https://gitlab.com/gitlab-org/gitlab-runner/issues/2226

I'm currently working on a gitlab runner that works locally.
Still in the early phases, but i'm developing fast since I have come to find it very very very useful.
It doesn't seem like gitlab want/have time to make this, so here you go.
https://github.com/firecow/gitlab-runner-local