CI & CD

This document exposes the steps to correctly configure the CD/CI environment integrated in gitlab server. The pipeline is runned in a dedicated computation server (not in the same machine where gitlab is installed). The run environment is embedded in a Docker image, where all the dependencies must be installed or should be installed on each test bench.

0. Pre-requisites

Before start, be sure that you have all what you need:

Gitlab server.
A testing server (called runner) to run the CD/CI pipelines.
Docker installed in the testing server.

1. Gitlab Certficates

The runner needs to download repo elements from the Gitlab server, so the Gitlab secure certificates must be included in the runner. To include the needed certificates:

Get the certs: Open the gitlab url in the browser, and then
- Click in the Padlock (left side of url textfield)
- Connection / Conexión Segura (click the right arrow)
- More information/Más informacion
- In the Security/Seguridad tab, View Certificate/Ver certificado
- Details/Detalles
- Export/Exportar
- Name the file correctly (see above).
Correct name of the cert file: The cert file should have the following naming .crt (e.g. url: https://gitlab.vicomtech.es/ -> cert: gitlab.vicomtech.es.crt)

2. Create "Test Runner" Docker Image

Define the content of the docker image using a Dockerfile. Take this Dockerfile as an example:

FROM ubuntu:16.04
MAINTAINER [email protected]
# install prereqs
RUN apt-get update &&\
    apt-get install -y curl vi apt-transport-https ca-certificates gnupg-agent software-properties-common &&\
    apt-get clean && rm -rf /tmp/* /var/tmp/* /var/cache/apt/archives/* /var/lib/apt/lists/*
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - &&\
   apt-key fingerprint 0EBFCD88 &&\
   add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" &&\
   apt-get update && apt-get install -y docker-ce docker-ce-cli containerd.io
RUN curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh |  bash 
RUN apt-get update && apt-get install  -y gitlab-runner &&\
    apt-get clean && rm -rf /tmp/* /var/tmp/* /var/cache/apt/archives/* /var/lib/apt/lists/*
RUN mkdir /etc/gitlab-runner/certs/
COPY <hostname>.crt /etc/gitlab-runner/certs/

Build the image

sudo docker build -f <name of the dockerfile> -t gitlab-runner:latest .

3. Run the container & Register runner to each gitlab project

We want to execute the test on the host server not in the gitlab-runner container. Thus, when running the gitlab-runner container, we need to bridge the docker.io sockets with the host server. cmd

sudo docker run  -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /gitlab-runner-config/:/etc/gitlab-runner  /bin/bash

Check project data in gitlab webpage to register the runner

Open projects gitlab webpage. Click on Settings-> CDCI -> Runner (Expand)
Check data in section: Specific Runners -> Set up a specific Runner manually

  gitlab-runner register

gitlab url
registration token
runner name
tag name
default docker image

Restart the runners

  gitlab-runner restart

Very important Issue
In gitlab CD/CI webpage (access via 1.), below Set up a specific Runner manually, a section called Runners activated for this project should appear. The registered runner might be stoped/paused, to make it running.
-Click on the :fa-edit: (box with a pencil) and then in Run untagged jobs activate
- Indicates whether this runner can pick jobs without tags**
To check registered runners

 cat /etc/gitlab-runner/config.toml

Create a CD pipelines Create a .gitlab.ci.yml file on the root folder. .gitlab.ci.yml file

stages:
#  - analysis
  - build
  - test
  
#cppcheck:
#  image: ubuntu:18.04
#  variables: 
#    GIT_SSL_NO_VERIFY: "true"
#  stage: analysis
#  before_script:
#    - apt update
#    - apt install -y --no-install-recommends cppcheck=1.82-1
#    - cppcheck --version
#  script:
#    - cppcheck library/ --verbose --enable=all --inconclusive --language=c++ --suppress=missingIncludeSystem --error-exitcode=1
#clang-tidy:
#  image: archlinux/base:latest  
#  stage: analysis
#  variables:
#    GIT_SSL_NO_VERIFY: "true"
#    CC: clang
#    CXX: clang++
#  before_script:
#    - pacman -Syu --needed --noconfirm clang-tools-extra=6.0.1-2
#    - clang-tidy --version
#  script:
#    - clang-tidy -warnings-as-errors="*" -checks="-*,clang-analyzer-*,-clang-analyzer-alpha*" lib/hello.cpp -- -Iinclude   
build_ubuntu1804:
    image: ubuntu1804_cmake_gcc_asl:latest
    variables: 
      GIT_SSL_NO_VERIFY: "true"
    stage: build
    before_script:
      - apt update
    script:
      - mkdir -p build && cd build
      - cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++11 -DADD_VCD_TRANS=ON -DBUILD_PERF=OFF -DBUILD_TESTS=ON -DBUILD_SAMPLES=ON -DENABLE_GUID=OFF --build . ../library
      - make
      - make install
build_arm:
    image: ubuntu1604_cmake_arm_asl:latest
    variables: 
      GIT_SSL_NO_VERIFY: "true"
    stage: build
    before_script:
      - apt update
    script:
      - mkdir -p build && cd build
      - cmake -DCMAKE_TOOLCHAIN_FILE=~/platforms/linux/arm-gnueabi.toolchain.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++11 -DADD_VCD_TRANS=ON -DBUILD_PERF=OFF -DBUILD_TESTS=ON -DBUILD_SAMPLES=ON -DENABLE_GUID=OFF --build . ../library
      - make 
      - make install 
test:
    image: ubuntu1804_cmake_gcc_asl:latest
    variables: 
      GIT_SSL_NO_VERIFY: "true"
    stage: test
    before_script:
      - apt update
    script:
      - mkdir -p build && cd build
      - cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS=-std=c++11 -DADD_VCD_TRANS=ON -DBUILD_PERF=OFF -DBUILD_TESTS=ON -DBUILD_SAMPLES=OFF -DENABLE_GUID=OFF --build . ../library
      - make && make install
      - bin/test_images
      - bin/test_objects
      - bin/test_asl

the image used for building and testing should be pregenerated with dockerfiles

jgoenetxea/ci_cd_gitlab.md

CI & CD

0. Pre-requisites

1. Gitlab Certficates

2. Create "Test Runner" Docker Image

3. Run the container & Register runner to each gitlab project