sjenning

Keybase proof

I hereby claim:

• I am sjenning on github.
• I am sjennings (https://keybase.io/sjennings) on keybase.
• I have a public key whose fingerprint is 6786 EF89 9CFF 4913 F200 2FE2 9622 D5C8 C1AC F915

To claim this, I am signing this object:

sjenning

$ curl http://127.0.0.1:8080/apis/catalog/v1/namespaces/default/catalogs
{
  "kind": "CatalogList",
  "apiVersion": "catalog/v1",
  "metadata": {
    "selfLink": "/apis/catalog/v1/namespaces/default/catalogs",
    "resourceVersion": "31"
  },
  "items": []
}

sjenning

Linux Hugepages

Hugepages are a hardware feature designed to reduce pressure on the translation lookaside buffer (TLB) for applications that operate on large contiguous memory regions.

Take a program that operates on a large 2MB internal structure as an example. If the program accesses that space in such a way that one byte in each regular 4k page is accessed, 2M/4k = 512 TLB entries are needed. Each TLB miss at the hardware level requires and interrupt and kernel intervention to resolve. However, if the allocation is backed by a 2M hugepage by mmap()ing with MAP_HUGETLB, only 1 TLB entry is required.

On x86_64, there are two hugepage sizes: 2MB and 1G. 1G hugepages are also called gigantic pages. 1G must be enabled on kernel boot line with hugepagesz=1G. Hugeages, especially 1G ones, should to be allocated early before memory fragments (i.e. at/near boot time) to increase the likelihood that they can be allocated successfully with minimal memory migration (i.e. defreg) required

sjenning

Kubernetes on a NUMA machine

Example Topology

2 scoket (2 node) machine with 4 cores/socket, 0-3 on node0 and 4-7 on node1

kubelet options

kubelet-node0

sjenning

[OSEv3:children]
masters
nodes

[OSEv3:vars]
ansible_ssh_user=sjennings
ansible_become=yes

deployment_type=origin

sjenning

manual (fedora 25)

nodes:
gluster-node1
gluster-node2

per node:
setenforce 0
dnf install glusterfs-server
systemctl enable --now glusterd.service

sjenning

setenforce 0
vi /etc/selinux/config (change to "permissive")
dnf install wget make gcc git docker bash-completion -y
systemctl enable --now docker
wget https://storage.googleapis.com/golang/go1.7.4.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.7.4.linux-amd64.tar.gz
vi ~/.bashrc
export GOROOT=/usr/local/go
export GOPATH=$HOME/go

sjenning

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: gluster-volume
  annotations:
    volume.beta.kubernetes.io/storage-class: default
spec:
  accessModes:
  - ReadWriteOnce
  resources:

sjenning

---
apiVersion: v1
kind: Secret
metadata:
  name: heketi-secret
  namespace: default
data:
  key: YWRtaW5zZWNyZXQ=
type: kubernetes.io/glusterfs
---

sjenning

#include <assert.h>
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <libgen.h>
#include <limits.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>