Skip to content

Instantly share code, notes, and snippets.

@mstaicu
Last active December 5, 2023 13:42
Show Gist options
  • Save mstaicu/54c05977f13df250abdf2314acf818ed to your computer and use it in GitHub Desktop.
Save mstaicu/54c05977f13df250abdf2314acf818ed to your computer and use it in GitHub Desktop.
Barebones NATS JetStream clustering in Kubernetes with PVC and no Helm charts
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nats
spec:
replicas: 3
selector:
matchLabels:
app: nats
# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-network-id
serviceName: nats-headless
template:
metadata:
labels:
app: nats
spec:
containers:
- name: nats
image: nats:alpine
args: ["-p", "4222", "-c", "/etc/nats/nats-server.conf"]
ports:
- name: nats-client
containerPort: 4222
- name: nats-cluster
containerPort: 6222
volumeMounts:
- name: nats-server-configmap
mountPath: /etc/nats/
- name: nats-pvc
mountPath: /data
volumes:
- name: nats-server-configmap
configMap:
name: nats-server-config-cm
volumeClaimTemplates:
- metadata:
name: nats-pvc
spec:
accessModes: ["ReadWriteOnce"]
# Consult your cloud provider with regards to available PVC classes
# otherwise dynamic provisioning will probably fail
storageClassName: do-block-storage
resources:
requests:
storage: 5Gi
---
# StatefulSets currently require a Headless Service to be responsible
# for the network identity of the Pods. You are responsible for creating this Service.
# https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#stable-network-id
apiVersion: v1
kind: Service
metadata:
name: nats-headless
spec:
# $ kubectl -n kube-system get svc kube-dns
# NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
# kube-dns ClusterIP 10.245.0.10 <none> 53/UDP,53/TCP,9153/TCP 4d22h
# $ kubectl get ep
# NAME ENDPOINTS AGE
# kubernetes 100.65.58.7:443 4d22h
# nats-headless 10.244.0.119:4222,10.244.0.195:4222,10.244.1.83:4222 + 3 more... 18m
# $ kubectl exec -it nats-0 -- /bin/ash
# # nslookup 10.244.0.119 10.245.0.10
clusterIP: None
selector:
app: nats
ports:
- name: nats
protocol: TCP
port: 4222
targetPort: 4222
- name: nats-cluster
protocol: TCP
port: 6222
targetPort: 6222
---
apiVersion: v1
kind: Service
metadata:
name: nats-lb-srv
spec:
type: LoadBalancer
selector:
app: nats
ports:
- name: http
protocol: TCP
port: 4222
targetPort: 4222
apiVersion: v1
kind: ConfigMap
metadata:
name: nats-server-config-cm
data:
# https://github.com/nats-io/nats-docker/blob/256811340a89b8a3b78f1a144b6a87fba4424b67/2.10.x/scratch/Dockerfile#L9
# https://docs.nats.io/running-a-nats-service/configuration/clustering/jetstream_clustering#raft
nats-server.conf: |
server_name=$HOSTNAME
accounts: {
$SYS: {
users: [
{ user: admin, password: password }
]
},
TMA: {
jetstream: enabled,
users: [
{ user: me, password: admin }
]
}
}
jetstream {
"store_dir": "/data"
}
cluster: {
name: EU,
port: 6222,
routes: [
"nats://nats-0.nats-headless:6222"
]
}
# Since the cluster uses a gossiping protocol to uncover other replicas, we don't need to specify
# the URL's for all seed servers in the routes array, we can keep only the first one
# The official deployment files however keep all the routes in the config
# routes: [
# "nats://nats-0.nats-headless:6222",
# "nats://nats-1.nats-headless:6222",
# "nats://nats-2.nats-headless:6222",
# ]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment