Skip to content

Instantly share code, notes, and snippets.

@steele-ntwrk
Last active November 18, 2023 00:02
Show Gist options
  • Select an option

  • Save steele-ntwrk/4c4b6d10c2cc79a5efd19ef219603d8b to your computer and use it in GitHub Desktop.

Select an option

Save steele-ntwrk/4c4b6d10c2cc79a5efd19ef219603d8b to your computer and use it in GitHub Desktop.
Describe the process of setting up CICD pipeline

Configuring a CI/CD Pipeline

Handy Commands

Update Linux - sudo apt-get update

Create Python virtual env python3 -m venv /path/to/new/virtual/environment

Activate Virtual Env source virtualenvs/netauto/bin/activate

Ansible located at etc/ansible/

Cisco command aaa authorization exec default local if-authenticated

Docker Volume location /var/snap/docker/common/var-lib-docker/volumes

Infrastructure and Software

Home Server: 16gb RAM i5 CPU

Host Software: VMware ESXi 7

Linux OS: Ubuntu 20.x

Virtual Network/Testbed: Cisco CML2

Network Lab Server: Ubuntu 20.2 Virtual Environment: netauto

Version Control and Artifact Repo: Github

Continuous Integration and Continuous Deployment (CI/CD): Jenkins

Deployment: Ansible and pyATS

Testing: pyATS/batfish

Network Source of Truth: Netbox

Container Manager: Docker

Network Templates: Jinja2

Programming Language Python Version: Python 3.9.7

Network Topology

This is my general network topology

Setting up CML VM

Setting up CML VM was relatively simple, I followed this guide by X . The only thing that tripped me up was getting my virtual devices connected to the outside network. To get external network access I required to create a bridge network in CML and then make sure the the host vSwitch is configured to accept some non-default security settings, see below.

Insert ScreenShot

Setting Up Linux VM

Setting up the VM was quite a trial and error in getting everything setup the way i would like. Im sure my design is not ideal since everything is sitting on one box but I wanted to keep it simple as possible and the ultimate goal is to docker all of this so i can pick it up and put it down where needed.

  1. Update Linux box sudo apt-get update

  2. Install pip libary so other packages can be installed sudo apt-get install python3-pip

  3. Install Virtual Env to establish a vitrual ENV sudo apt install python3.9-venv

  4. Create Virtual env python3 -m venv /home/netauto/virtualenvs/netauto

  5. Generate a SSH Key to import into github ssh-keygen -t rsa -b 4096 -C "steele.ntwrk@gmail.com"

  6. Copy key from location and add to github

  7. Install Jinja2 sudo apt-get install python-jinja2 in venv

  8. Install netbox custom image using docker, with plugins nextbox-ui-plugin and netbox-qrcode:

    1. Copy netbox-docker repo git clone -b release https://github.com/netbox-community/netbox-docker.git

    2. Create files in base files netbox directory

    touch plugin_requirements.txt
    touch Dockerfile-Plugins
    touch docker-compose.override.yml
    
    1. Edit files for required plugins

      pico plugin_requirements.txt
      nextbox-ui-plugin
      netbox-qrcode
      
      • Define plugins in netbox configuration
          pico configuration/configuragtion.py
      PLUGINS = [
          'nextbox_ui_plugin','netbox_qrcode'
      ]
      • Create plugin docker file
      pico Dockerfile-Plugins
      FROM netboxcommunity/netbox:latest
      
      COPY ./plugin_requirements.txt /
      RUN /opt/netbox/venv/bin/pip install  --no-warn-script-location -r /plugin_requirements.txt
      
      # These lines are only required if your plugin has its own static files.
      COPY configuration/configuration.py /etc/netbox/config/configuration.py
      RUN SECRET_KEY="dummy" /opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py collectstatic --no-input
      • Edit docker-compose.overide.yml
      version: '3.4'
      services:
      netbox:
          restart: unless-stopped
          ports:
          - 8000:8080
          build:
          context: .
          dockerfile: Dockerfile-Plugins
          image: netbox:latest-plugins
      netbox-worker:
          restart: unless-stopped
          image: netbox:latest-plugins
      netbox-housekeeping:
          restart: unless-stopped
      postgres:
          restart: unless-stopped
      redis:
          restart: unless-stopped
      redis-cache:
          restart: unless-stopped
      • Build Docker and Deploy
    docker-compose build --no-cache
    docker-compose up -d
    
    - Setup NAPALM Credentials
    
    sudo pico netbox-docker/env/netbox.env
    NAPALM_USERNAME=admin
    NAPALM_PASSWORD=password
    
  9. Import device models using Github Tool, install into virtual env

git clone https://github.com/minitriga/Netbox-Device-Type-Library-Import 
cd Netbox-Device-Type-Library-Import
pip install -r requirements.txt
cp .env.example .env
pico .env"
# Untested
NETBOX_URL=environ.get(NETBOX_URL)
NETBOX_TOKEN=environ.get(NETBOX_TOKEN)
  1. Install ansible
sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

-Configure ansible modify ansible.cfg with https://github.com/ansible/ansible/blob/stable-2.10/examples/ansible.cfg and set hostkey to False

test with ansible all -i 10.10.0.133, -c network_cli -u admin -k -m ios_facts -e ansible_network_os=ios

  1. Install Gitlab create folder

create docker compose file

version: '3.6'
services:
 web:
   image: 'gitlab/gitlab-ce:latest'
   container_name: gitlab
   restart: unless-stopped
   environment:
     GITLAB_OMNIBUS_CONFIG: |
       external_url 'http://gitlab.example.com'
       # Add any other gitlab.rb configuration here, each on its own line
   ports:
     - '8080:80'
     - '8929:8929'
     - '2224:22'
   volumes:
     - 'gitlab_config:/etc/gitlab'
     - 'gitlab_logs:/var/log/gitlab'
     - 'gitlab_data:/var/opt/gitlab'
   shm_size: '2gb'

volumes:
 gitlab_config: {}
 gitlab_logs: {}
 gitlab_data: {}

up docker images

reset root password

sudo docker exec -it 3b08763d922b bash
gitlab-rake "gitlab:password:reset[root]"

git clone git@10.10.0.144:2224/netauto/configuration-backup.git

[user] name = netauto email = email

[includeIf "gitdir:~/config-backup"] path = .gitconfig_work

Candidate Environment Variables

sudo pico ~/.bashrc
#Netbox Service Account
export NAPALM_USERNAME=admin
export NAPALM_PASSWORD=password
export NETBOX_URL="http://10.10.0.144:8000/"
export NETBOX_TOKEN="0123456789abcdef0123456789abcdef01234567"

NAPALM_ARGS = { 'secret': NAPALM_PASSWORD, 'enable_password': NAPALM_PASSWORD # Include any additional args here }

Netbox inventory REPO_URL = the repo to look for device types (defaults to https://github.com/netbox-community/devicetype-library.git) REPO_BRANCH= the branch to check out if appropriate, defaults to master. NETBOX_URL="http://10.10.0.144:8000/" NETBOX_TOKEN="0123456789abcdef0123456789abcdef01234567" VENDORs="Cisco,Juniper,Generic"

Links to websites:

-https://github.com/netbox-community/netbox-docker/wiki/Using-Netbox-Plugins

-https://linuxhint.com/ssh_key_setup_github/

-https://docs.python.org/3/library/venv.html

-https://www.jenkins.io/doc/book/installing/linux/#prerequisites

-https://networkotaku.wordpress.com/2017/06/06/network-automation-template-configurations-with-jinja2-and-yaml/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment