MongoDB statefulset for kubernetes with authentication and replication

mongo-statefulset.yaml

## Generate a key
# openssl rand -base64 741 > mongodb-keyfile
## Create k8s secrets
# kubectl create secret generic mongo-key --from-file=mongodb-keyfile
---
apiVersion: v1
kind: Service
metadata:
  name: mongo
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    role: mongo
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: "mongo"
  replicas: 1
  template:
    metadata:
      labels:
        role: mongo
        environment: test
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo:3.4.9
          command:
          - /bin/sh
          - -c
          - >
            if [ -f /data/db/admin-user.lock ]; then
              mongod --replSet rs0 --clusterAuthMode keyFile --keyFile /etc/secrets-volume/mongodb-keyfile --setParameter authenticationMechanisms=SCRAM-SHA-1;
            else
              mongod --auth;
            fi;
          lifecycle:
            postStart:
              exec:
                command:
                - /bin/sh
                - -c
                - >
                  if [ ! -f /data/db/admin-user.lock ]; then
                    sleep 5;
                    touch /data/db/admin-user.lock
                    if [ "$HOSTNAME" = "mongo-0" ]; then
                      mongo --eval 'db = db.getSiblingDB("admin"); db.createUser({ user: "admin", pwd: "password", roles: [{ role: "root", db: "admin" }]});';
                    fi;
                    mongod --shutdown;
                  fi;
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-key
              mountPath: "/etc/secrets-volume"
              readOnly: true
            - name: mongo-persistent-storage
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: "role=mongo,environment=test"
            - name: MONGODB_USERNAME
              value: admin
            - name: MONGODB_PASSWORD
              value: password
            - name: MONGODB_DATABASE
              value: admin
      volumes:
      - name: mongo-key
        secret:
          defaultMode: 0400
          secretName: mongo-key
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-storage
      annotations:
        volume.beta.kubernetes.io/storage-class: "fast"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 100Gi

Saved my life thanks a lot!!!!

Works great, when replicas: 1. But, when I try to make replicas: 2, and login to the 2nd mongo pod, I see it as rs0:OTHER> instead of the expected secondary mongo pod rs0:SECONDARY>
Looks like this approach is having difficulty in creating the secondary mongo pod in replica set.
It will be a great help if you could let me know how to do this?
Thanks,
Amit

Because you haven't initialized Replication Set at mongo using rs.initiate on this example

@thilinapiy man its been a long time. Just came across this and I'm going to use it on my own project. Great work mate.

Hi, I'm new to the Database, I have executed the above mongo with 3 replicaset and created successfully in kubernetes managed environment.

When I exected the rs.initiate it failed with HostUnreachable can anyone help here pls

rs.initiate(
... {
... _id: "rs0",
... version: 1,
... members: [
... { _id: 0, host : "mongo-0:27017" },
... { _id: 1, host : "mongo-1:27017" },
... { _id: 2, host : "mongo-2:27017" }
... ]
... }
... )
{
"ok" : 0,
"errmsg" : "replSetInitiate quorum check failed because not all proposed set members responded affirmatively: mongo-1:27017 failed with HostUnreachable, mongo-2:27017 failed with HostUnreachable",
"code" : 74,
"codeName" : "NodeNotFound"

I copy-pasted this file exactly as it is. I created the secret as well like the commented section says, I am still getting below error:

2019-05-28T10:05:14.173+0000 I NETWORK  [conn208] received client metadata from 127.0.0.1:35862 conn208: { driver: { name: "nodejs", version: "2.2.36" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.15.0-1040-azure" }, platform: "Node.js v11.2.0, LE, mongodb-core: 2.1.20" }
2019-05-28T10:05:14.174+0000 I ACCESS   [conn208] SCRAM-SHA-1 authentication failed for admin on admin from client 127.0.0.1:35862 ; UserNotFound: Could not find user admin@admin```

Please help

I copy-pasted this file exactly as it is. I created the secret as well like the commented section says, I am still getting below error:

It works for me, even automatically initialization by side car, from your log command mongo --eval 'db = db.getSiblingDB("admin"); db.createUser({ user: "admin", pwd: "password", roles: [{ role: "root", db: "admin" }]});';

was for some reason not executed,

Guys this is old now. There are better ways to do it.
Tryout operators.

@thilinapiy Is MongoDB Enterprise Kubernetes Operator free to use in development?

Unable to mount volumes for pod "mongo-0_dev(xxx-xx-xx-xx-xx)": timeout expired waiting for volumes to attach or mount for pod "dev"/"mongo-0". list of unmounted volumes=[mongo-key]. list of unattached volumes=[mongo-persistent-storage mongo-key default-token-nvw6d]

how your creating mongodb-keyfile.

Kindly help me to resolve in this issue.

@sandyvanam, it's mentioned at top of the YAML.

## Generate a key
# openssl rand -base64 741 > mongodb-keyfile
## Create k8s secrets
# kubectl create secret generic mongo-key --from-file=mongodb-keyfile

Works ok, Thanks You safe my life!!

Hi Guys, I am also using the Statefulset, need help while I am trying to login to mongo-0 pod (kubectl exec -ti mongo-0 mongo) and trying to create new user but I am getting below error.

rs0:PRIMARY> use admin
switched to db admin
rs0:PRIMARY> db.createUser({user:"replSetManager",pwd:"password",roles:[{role:"clusterManager",db:"admin"},{role:"dbOwner", db:"adminsblog"},{role:"readWrite", db:"departmentblog"},{role:"read", db:"otherblog"}]})
2021-02-15T07:15:47.812+0000 E QUERY [thread1] Error: couldn't add user: not authorized on admin to execute command { createUser: "replSetManager", pwd: "xxx", roles: [ { role: "clusterManager", db: "admin" }, { role: "dbOwner", db: "adminsblog" }, { role: "readWrite", db: "departmentblog" }, { role: "read", db: "otherblog" } ], digestPassword: false, writeConcern: { w: "majority", wtimeout: 600000.0 } } :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DB.prototype.createUser@src/mongo/shell/db.js:1292:15
@(shell):1:1

@lalit1980 did you come right with the above error? how did you fix it?

@thilinapiy Is MongoDB Enterprise Kubernetes Operator free to use in development?

It's free for development not for commercial

@thilinapiy the Community version is here https://github.com/mongodb/mongodb-kubernetes-operator

@thilinapiy By any chance, do you know how to add the internal authentication using the MongoDB enterprise kubernetes operator with the keyfile. I do not find much information on the crds and haven't got any good examples in the git repository.