Multus: Reference external namespace in openshift

README.md

Referring to "global namespaces" using Multus CNI in OpenShift

This demonstrates using a cross-namespace reference in OpenShift to refer to net-attach-defs in the openshift-multus namespace from another namespace.

See the additional pod.yml and net-attach-def.yaml files included in this gist.

Using latest OCP from CI (4.9 master)

[doug@home]$ oc get nodes
NAME                                       STATUS    ROLES     AGE       VERSION
ci-ln-670xzb2-f76d1-km9ss-master-0         Ready     master    97m       v1.21.1+8268f88
ci-ln-670xzb2-f76d1-km9ss-master-1         Ready     master    97m       v1.21.1+8268f88
ci-ln-670xzb2-f76d1-km9ss-master-2         Ready     master    96m       v1.21.1+8268f88
ci-ln-670xzb2-f76d1-km9ss-worker-a-6sr5q   Ready     worker    90m       v1.21.1+8268f88
ci-ln-670xzb2-f76d1-km9ss-worker-b-v2cdp   Ready     worker    89m       v1.21.1+8268f88
ci-ln-670xzb2-f76d1-km9ss-worker-c-pgwgh   Ready     worker    90m       v1.21.1+8268f88

Create the net-attach-def in project multus.

[doug@home]$ oc project openshift-multus
[doug@home]$ oc create -f nad.yml 
networkattachmentdefinition.k8s.cni.cncf.io/macvlan-conf created
[doug@home]$ oc get net-attach-def
NAME           AGE
macvlan-conf   9s

Create a new project.

[doug@home]$ oc new-project dougtest
Now using project "dougtest" on server "https://api.ci-ln-670xzb2-f76d1.origin-ci-int-gce.dev.openshift.com:6443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git

to build a new example application in Ruby.
[doug@home]$ oc get net-attach-def
No resources found.

Note the net-attach-def is not shown in this namespace.

Create the pod. And inspect the interfaces to see that it was launched and is using Multus CNI (note: net1 interface in ip a)

[doug@home]$ oc project dougtest
Already on project "dougtest" on server "https://api.ci-ln-670xzb2-f76d1.origin-ci-int-gce.dev.openshift.com:6443".
[doug@home]$ nano pod.yml
[doug@home]$ oc create -f pod.yml 
pod/samplepod created
[doug@home]$ watch -n1 oc get pods
[doug@home]$ oc exec -it samplepod -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: eth0@if61: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1410 qdisc noqueue state UP 
    link/ether 0a:58:0a:83:00:38 brd ff:ff:ff:ff:ff:ff
    inet 10.131.0.56/23 brd 10.131.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::743c:22ff:fec1:73f6/64 scope link 
       valid_lft forever preferred_lft forever
4: net1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP 
    link/ether 32:91:6a:06:51:56 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.200/24 brd 192.168.1.255 scope global net1
       valid_lft forever preferred_lft forever
    inet6 fe80::3091:6aff:fe06:5156/64 scope link 
       valid_lft forever preferred_lft forever

net-attach-def.yaml

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf
spec:
  config: '{
      "cniVersion": "0.3.0",
      "type": "macvlan",
      "master": "ens4",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.1.0/24",
        "rangeStart": "192.168.1.200",
        "rangeEnd": "192.168.1.216",
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "gateway": "192.168.1.1"
      }
    }'

pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: samplepod
  annotations:
    k8s.v1.cni.cncf.io/networks: openshift-multus/macvlan-conf
spec:
  containers:
  - name: samplepod
    command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: alpine