kubernetes-ingress websockets with nodejs

frontend-ws-connection.ts

export const ws = webSocket<WebsocketMessage>(`wss://${location.hostname}:${location.protocol === 'https:' ? 443 : 80}/ws/`);

export const wsObserver = ws
  .pipe(
    retryWhen(errors =>
      errors.pipe(
        delay(1000)
      )
    )
  );

wsObserver.subscribe(console.log);

ingress-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  namespace: <YOUR_NAMESPACE>
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
    nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: client-cluster-ip-service
              servicePort: 3000
          # Below is the important part!
          - path: /ws/
            backend:
              serviceName: server-cluster-ip-service
              servicePort: 40510

server-cluster-ip-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
  namespace: <YOUR_NAMESPACE>
spec:
  type: ClusterIP
  selector:
    component: server
  ports:
    - port: 40510
      targetPort: 40510
      # The below line isn't required.
      protocol: TCP

server-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: server-deployment
  namespace: <YOUR_NAMESPACE>
spec:
  replicas: 1
  selector:
    matchLabels:
      component: server
  template:
    metadata:
      labels:
        component: server
    spec:
      containers:
        - name: server
          image: <YOUR_DOCKER_IMAGE>
          ports:
            - containerPort: 40510

server.js

import { Server as WebSocketServer } from 'ws';

// IMPORTANT: not a secure connection
const wss = new WebSocketServer({
    path: '/ws/',
    port: 40510,
});

wss.on('connection', function (ws) {
    console.log('connection!');
});

wss.on('close', function close() {
    console.log('ws disconnected');
});

If you are using kubernetes you should probably instlal https://github.com/jetstack/cert-manager and update your ingress to handle SSL certificate.

If you are using kubernetes you should probably instlal https://github.com/jetstack/cert-manager and update your ingress to handle SSL certificate.

I have SSL terminating at the ingress controller, with a certificate configured. Do I also need to configure an additional certificate on the backend service?

Thanks @jsdevtom!!!
I just had the issue with the websocket connection close after 30 seconds and therefore I find your gist.

I'm using GKE with the L7 GLB. For those, who are using the Google cloud infrastructure it is pretty simple to get websocket connections working with google managed ssl certificates. It looks a little bit different:

apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
  name: service-backend-config
spec:
  timeoutSec: 3600

---

apiVersion: v1
kind: Service
metadata:
  labels:
    app: service
  name: service
  annotations:
    cloud.google.com/backend-config: '{"ports": {"8080":"service-backend-config"}}'
spec:
  ports:
    - name: http
      port: 8080
      protocol: TCP
      targetPort: 8080
  selector:
    app: service
  type: NodePort

---

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: public-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "your-ip"
    networking.gke.io/managed-certificates: "your-certificates"
spec:
  rules: 
  - host: "your-domain.com"
    http:
      paths:
      - backend:
          serviceName: service
          servicePort: 8080
        path: /*

in k8s 1.18.8 the annotations need a string not a number:
nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
==>
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"

still not working though

I'm sure that missing trailing forward slash on /ws/ would have cost me 10 hours too. So, thanks!

I cant get this to work with socket io and typescript. Any suggestion will be highly appreciated

After some hours trying to fix it, i found the proper config for my ingress ( k8s azure )

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-service
  namespace: development
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  rules:
    - host: svc-host-domain.com
      http:
        paths:
        - backend:
            service:
              name: my-service
              port:
                number: 8084
          path: /publish
          pathType: Prefix
        - backend:
            service:
              name: my-service
              port:
                number: 8080
          path: /ws/
          pathType: Prefix

Thank you for the great snippet

( i'm still struggling to have the keep-alive working well, it disconnects my websocket clients suddenly )

I'm sure that missing trailing forward slash on /ws/ would have cost me 10 hours too. So, thanks!

What should happen if the trailing slash is missing? I'm trying to debug an error related to my k8s websocket setup, and I'm using /ws right now, but I'm not exactly sure it's failing...

A working e2e complete setup of this scenario is available here: https://github.com/martencassel/nodejs-websocket-nginx-ingress-setup

A working e2e complete setup of this scenario is available here: https://github.com/martencassel/nodejs-websocket-nginx-ingress-setup

🔥

Thank you you save me a lot of time. I have been struggling with this issue, at the time I didn't know that I have to configure my ingress for websocket. Thank you again.

Thank you for this, was struggling with something very similar when trying to use an ingress for my websocket client when deployed in minikube. If it helps anyone else:

Websocket client:

const websocketUrl = "ws://minikube.local/ws/"; // Replace with your WebSocket server URL
const socket = new WebSocket(websocketUrl);

(for local deployment, edit /etc/hosts so that minikube.local resolves to your minikube ip)

Websocket-server ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: websocket-server-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: minikube.local
    http:
      paths:
      - path: /ws/
        pathType: Prefix
        backend:
          service:
            name: websocket-server
            port:
              number: 8080

Websocket-server service:

apiVersion: v1
kind: Service
metadata:
  name: websocket-server
spec:
  selector:
    app: websocket-server
  ports:
  - protocol: TCP
    port: 8080