Skip to content

Instantly share code, notes, and snippets.

@dln
Last active December 28, 2022 12:54
Show Gist options
  • Save dln/f742e03b2a90204f3ca4f57d8445d1e1 to your computer and use it in GitHub Desktop.
Save dln/f742e03b2a90204f3ca4f57d8445d1e1 to your computer and use it in GitHub Desktop.
Daemonset to automatically stripe all local ssd disks and mount for use in GKE workloads.
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: local-ssd-init
labels:
app: local-ssd-init
spec:
template:
spec:
hostPID: true
nodeSelector:
cloud.google.com/gke-local-ssd: "true"
volumes:
- name: setup-script
configMap:
name: local-ssd-init
- name: host-mount
hostPath:
path: /tmp/setup
initContainers:
- name: local-ssd-init
image: marketplace.gcr.io/google/ubuntu1804
securityContext:
privileged: true
volumeMounts:
- name: setup-script
mountPath: /tmp
- name: host-mount
mountPath: /host
command:
- /bin/bash
- -c
- |
set -ex
cp /tmp/setup.sh /tmp/wait.sh /host/
/usr/bin/nsenter -m/proc/1/ns/mnt -- chmod u+x /tmp/setup/wait.sh /tmp/setup/setup.sh
/usr/bin/nsenter -m/proc/1/ns/mnt /bin/bash /tmp/setup/wait.sh
/usr/bin/nsenter -m/proc/1/ns/mnt /bin/bash /tmp/setup/setup.sh
containers:
- image: "gcr.io/google-containers/pause:2.0"
name: pause
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
commonLabels:
app.kubernetes.io/name: local-ssd-init
resources:
- daemonset.yaml
configMapGenerator:
- name: local-ssd-init
files:
- setup.sh
- wait.sh
#!/bin/bash
set -exo pipefail
isMounted() { findmnt -rno SOURCE,TARGET "$1" >/dev/null; }
SSDS=(`readlink -f /dev/disk/by-id/google-local-ssd-*`)
apt-get -y update && apt-get -y install mdadm xfsprogs --no-install-recommends
if [ -b "/dev/md0" ]; then
if isMounted "/dev/md0"; then
echo "Array already created and mounted. Exiting"
exit 0
fi
mdadm --stop /dev/md0
mdadm --zero-superblock "${SSDS[@]}"
fi
umount "${SSDS[@]}" 2>/dev/null || true
yes | mdadm --create /dev/md0 --force --level=0 "--raid-devices=${#SSDS[@]}" "${SSDS[@]}" || true
mkfs.xfs -f /dev/md0
mkdir -p /mnt/disks/ssd
mount /dev/md0 /mnt/disks/ssd
# This file can be used as a Volume to ensure only a prepared host will allow pods to be scheduled on it.
touch /mnt/disks/ssd/.initialized
#!/bin/bash
while sudo fuser /var/{lib/{dpkg,apt/lists},cache/apt/archives}/lock >/dev/null 2>&1; do sleep 1; done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment