Skip to content

Instantly share code, notes, and snippets.

@tillkuhn
Created November 22, 2016 17:41
Show Gist options
  • Save tillkuhn/c2dba43d60d3e7a2462928812ca9e4bf to your computer and use it in GitHub Desktop.
Save tillkuhn/c2dba43d60d3e7a2462928812ca9e4bf to your computer and use it in GitHub Desktop.
Kubernetes configuration file to create and expose Nginx Ingress Controller Service with sticky sessions, virtual host stats and default backend
---
################################################################################
## K8S Default Backend for Nginx if no endpoint is available e.g. 404 servers
###############################################################################
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-default-backend
namespace: kube-system
labels:
app: nginx-default-backend
spec:
replicas: 1
selector:
matchLabels:
app: nginx-default-backend
template:
metadata:
labels:
app: nginx-default-backend
group: lb
spec:
terminationGracePeriodSeconds: 60
containers:
- name: defaultbackend
# Any image is permissable as long as: 1. It serves a 404 page at /
# and 2. It serves 200 on a /healthz endpoint
image: gcr.io/google_containers/defaultbackend:1.0
imagePullPolicy: IfNotPresent
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 30
timeoutSeconds: 5
ports:
- containerPort: 8080
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
---
################################################################################
## K8S Service configuration default backend. in NGINX Deployment config
## use arg --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend
################################################################################
apiVersion: v1
kind: Service
metadata:
name: nginx-default-backend
namespace: kube-system
labels:
app: nginx-default-backend
group: lb
spec:
type: NodePort
ports:
- port: 8080
targetPort: 8080
selector:
app: nginx-default-backend
---
##################################################################################################
## K8S config map for NGINX LB Controller. supply as arg in deployment config
## - --nginx-configmap=$(POD_NAMESPACE)/nginx-ingress-lb-cfg
## See link below for all config options
## https://github.com/kubernetes/contrib/blob/master/ingress/controllers/nginx/configuration.md
###################################################################################################
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-ingress-lb-cfg
namespace: kube-system
labels:
app: nginx-ingress-lb
group: lb
data:
enable-sticky-sessions: 'true' ## use ROUTE cookie to provide session affinity
enable-vts-status: 'true' ## Allows the replacement of the default status page nginx-module-vts
---
############################################################################################
## K8S deplox config for NGINX LB gcr.io/google_containers/nginx-ingress-controller
## https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx
#########################################################################################
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
namespace: kube-system
labels:
app: nginx-ingress-lb
group: lb
spec:
replicas: 1
selector:
matchLabels:
app: nginx-ingress-lb
template:
metadata:
labels:
app: nginx-ingress-lb
name: nginx-ingress-lb
group: lb
spec:
terminationGracePeriodSeconds: 60
containers:
- name: nginx-ingress-lb
image: gcr.io/google_containers/nginx-ingress-controller:0.8.3
imagePullPolicy: IfNotPresent
readinessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
livenessProbe:
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
timeoutSeconds: 1
# use downward API
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 80
hostPort: 80
## if you terminate SSL at the AWS ELB you don't need port 443 here
- containerPort: 18080 ## we expose 18080 to access nginx stats in url /nginx-status
hostPort: 18080
## https://github.com/kubernetes/contrib/issues/1662 --watch-namespace to limit on one namespace
args:
- /nginx-ingress-controller
- --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend
- --nginx-configmap=$(POD_NAMESPACE)/nginx-ingress-lb-cfg
---
#######################################
# K8S NGINX LB Controller Service
#######################################
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-lb
namespace: kube-system
labels:
app: nginx-ingress-lb
group: lb
annotations:
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "YOUR_AWS_CERT_ID"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
spec:
type: LoadBalancer
ports:
- port: 443
targetPort: 80 ## terminate ssl
name: https
protocol: TCP
#- port: 80 ### disable unless you have http->https redirect or really want to support http besides https
# targetPort: 80
# name: http
# protocol: TCP
- port: 18080
targetPort: 18080
name: nginxstatus
protocol: TCP
selector:
app: nginx-ingress-lb
---
############################################################
## K8S Ingress to access Nginx status page from LB
############################################################
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress-lb-stats
namespace: kube-system
labels:
app: nginx-ingress-lb
group: lb
stage: kube-system
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: your.host.com
http:
paths:
- path: /nginx_status
backend:
serviceName: nginx-ingress-lb
servicePort: 18080
@chawkins88
Copy link

This was a huge help to me. The part with the ConfigMap was the missing link... Wanted to say thanks!

@bmarks-mylo
Copy link

bmarks-mylo commented Dec 29, 2016

So with an additional service (say a node.js deployment), does the nginx controller provide sticky sessions to that service's pods? I'm trying to achieve affinity to pods for websocket purposes.

Also, can you describe how traffic gets into the system? Is it Client -> ELB -> Nginx Ingress Controller Service -> Nginx Ingress Controller plus ingress rules -> MyApp Service -> MyApp Pod #N

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment