Skip to content

Instantly share code, notes, and snippets.

@dougbtv
Created August 11, 2021 16:09

Revisions

  1. dougbtv created this gist Aug 11, 2021.
    78 changes: 78 additions & 0 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,78 @@
    ## Referring to "global namespaces" using Multus CNI in OpenShift

    This demonstrates using a cross-namespace reference in OpenShift to refer to net-attach-defs in the `openshift-multus` namespace from another namespace.

    See the additional `pod.yml` and `net-attach-def.yaml` files included in this gist.


    Using latest OCP from CI (4.9 master)

    ```
    [doug@home]$ oc get nodes
    NAME STATUS ROLES AGE VERSION
    ci-ln-670xzb2-f76d1-km9ss-master-0 Ready master 97m v1.21.1+8268f88
    ci-ln-670xzb2-f76d1-km9ss-master-1 Ready master 97m v1.21.1+8268f88
    ci-ln-670xzb2-f76d1-km9ss-master-2 Ready master 96m v1.21.1+8268f88
    ci-ln-670xzb2-f76d1-km9ss-worker-a-6sr5q Ready worker 90m v1.21.1+8268f88
    ci-ln-670xzb2-f76d1-km9ss-worker-b-v2cdp Ready worker 89m v1.21.1+8268f88
    ci-ln-670xzb2-f76d1-km9ss-worker-c-pgwgh Ready worker 90m v1.21.1+8268f88
    ```

    Create the net-attach-def in project multus.

    ```
    [doug@home]$ oc project openshift-multus
    [doug@home]$ oc create -f nad.yml
    networkattachmentdefinition.k8s.cni.cncf.io/macvlan-conf created
    [doug@home]$ oc get net-attach-def
    NAME AGE
    macvlan-conf 9s
    ```

    Create a new project.

    ```
    [doug@home]$ oc new-project dougtest
    Now using project "dougtest" on server "https://api.ci-ln-670xzb2-f76d1.origin-ci-int-gce.dev.openshift.com:6443".
    You can add applications to this project with the 'new-app' command. For example, try:
    oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git
    to build a new example application in Ruby.
    [doug@home]$ oc get net-attach-def
    No resources found.
    ```

    Note the net-attach-def is not shown in this namespace.

    Create the pod. And inspect the interfaces to see that it was launched and is using Multus CNI (note: `net1` interface in `ip a`)

    ```
    [doug@home]$ oc project dougtest
    Already on project "dougtest" on server "https://api.ci-ln-670xzb2-f76d1.origin-ci-int-gce.dev.openshift.com:6443".
    [doug@home]$ nano pod.yml
    [doug@home]$ oc create -f pod.yml
    pod/samplepod created
    [doug@home]$ watch -n1 oc get pods
    [doug@home]$ oc exec -it samplepod -- ip a
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
    valid_lft forever preferred_lft forever
    3: eth0@if61: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1410 qdisc noqueue state UP
    link/ether 0a:58:0a:83:00:38 brd ff:ff:ff:ff:ff:ff
    inet 10.131.0.56/23 brd 10.131.1.255 scope global eth0
    valid_lft forever preferred_lft forever
    inet6 fe80::743c:22ff:fec1:73f6/64 scope link
    valid_lft forever preferred_lft forever
    4: net1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP
    link/ether 32:91:6a:06:51:56 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.200/24 brd 192.168.1.255 scope global net1
    valid_lft forever preferred_lft forever
    inet6 fe80::3091:6aff:fe06:5156/64 scope link
    valid_lft forever preferred_lft forever
    ```
    21 changes: 21 additions & 0 deletions net-attach-def.yaml
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,21 @@
    apiVersion: "k8s.cni.cncf.io/v1"
    kind: NetworkAttachmentDefinition
    metadata:
    name: macvlan-conf
    spec:
    config: '{
    "cniVersion": "0.3.0",
    "type": "macvlan",
    "master": "ens4",
    "mode": "bridge",
    "ipam": {
    "type": "host-local",
    "subnet": "192.168.1.0/24",
    "rangeStart": "192.168.1.200",
    "rangeEnd": "192.168.1.216",
    "routes": [
    { "dst": "0.0.0.0/0" }
    ],
    "gateway": "192.168.1.1"
    }
    }'
    11 changes: 11 additions & 0 deletions pod.yml
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,11 @@
    apiVersion: v1
    kind: Pod
    metadata:
    name: samplepod
    annotations:
    k8s.v1.cni.cncf.io/networks: openshift-multus/macvlan-conf
    spec:
    containers:
    - name: samplepod
    command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: alpine