Eric Ernst egernst

Running Openshift on top of Clear Containers 3.0

Overview

This documents how to get Openshift 3.6 on top of Kubernetes running with Clear Containers. An ansible playbook is used to provision a Fedora server 25 machine. Clear Containers is then installed and a basic test of deploying Clear Container and runc based pods is detailed.

Openshift

OpenShift is a public cloud application development and hosting platform which builds on top of the orchestration provided by Kubernetes.

Get go 1.8.3:

wget https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz
sudo tar -xvf go1.8.3.linux-amd64.tar.gz -C /usr/local/
mkdir -p $HOME/go/src
export GOPATH=$HOME/go
export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin
go version

Physical Setup:

---------------------
| workstation  [enp5s0f0] -- internet
|                  |          ---------------
|              [enp5s0f1] -- |    switch    |
--------------------         |              |
               [lab-nuc-1] --|              |
 [lab-nuc-2] --|--------------|

Using a Docker CNM plugin to play with SRIO-V

This gist describes the setup necessary for testing SRIO-V based connectivity between two physical boxes which are each setup as described here, and directly connected via their respective SRIO-V enabled NICs.

Setup host system's packages

For this scenario, I'm setting up two Ubuntu 16.04 systems which have a SRIO-V enabled interface as well as a second port for accessing the SUT. To setup:

Using a Docker CNM plugin to play with SRIO-V

This gist describes the setup necessary for testing SRIO-V based connectivity between two physical boxes which are each setup as described here, and directly connected via their respective SRIO-V enabled NICs.

Setup host system's packages

For this scenario, I'm setting up two Ubuntu 16.04 systems which have a SRIO-V enabled interface as well as a second port for accessing the SUT. To setup:

Basic VPP Setup for vhost-user interfaces and VMs

There are many examples of using VPP with tap interfaces/namespaces. After reviewing these, and details at https://wiki.fd.io/view/VPP/Tutorial_Routing_and_Switching, we came up with the following setup for testing inter-VM connectivity using VPP's vhost-user.

First time setup

Grab VPP packages, per directions from FD.io at https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages

$ sudo rm /etc/apt/sources.list.d/99fd.io.list
$ echo "deb [trusted=yes] https://nexus.fd.io/content/repositories/fd.io.ubuntu.xenial.main/ ./" | sudo tee -a /etc/apt/sources.list.d/99fd.io.list

Get the necessary sources

For plugin: go get github.com/clearcontainers/ovsdpdk
For ovs-dpdk enabled CC runtime: go get github.com/01org/cc-oci-runtime --> detail out getting the branch

Build the runtime

Will likely need a number of packages. For fedora 25, for example, run:

sudo dnf install \
glib2-devel-2.50.3-1.fc25.x86_64 \

Reproducing Memory issues with PC and OVS/DPDK

setup OVS

This assumes you already have OVS and DPDK installed on your system

sudo mkdir -p /var/run/openvswitch
sudo killall ovsdb-server ovs-vswitchd
rm -f /var/run/openvswitch/vhost-user*
sudo rm -f /etc/openvswitch/conf.db

export DB_SOCK=/var/run/openvswitch/db.sock

	#!/bin/bash

	go_src=$GOPATH/src

	find $go_src -name "*.go" -print > cscope.files

	if cscope -b -k; then
	echo "Done"
	else
	echo "Failed"

	Example docker file to create an image with a few tools and DPDK installed can be found below.

	This will be updated with usage with VPP CNM plugin and Clear Containers

	```
	FROM ubuntu:16.04

	ARG DPDK_VER=17.02
	ARG DPDK_TARGET=x86_64-native-linuxapp-gcc