Last active
May 27, 2019 14:33
-
-
Save kocolosk/d4bed1a993c0c506b1e58274352b30df to your computer and use it in GitHub Desktop.
CouchDB 2.0 in Kubernetes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Start a 3 node cluster and join it together automatically. Uses | |
# local ephemeral disk for database storage. | |
apiVersion: extensions/v1beta1 | |
kind: Deployment | |
metadata: | |
name: couchdb | |
spec: | |
replicas: 3 | |
template: | |
metadata: | |
labels: | |
run: couchdb | |
spec: | |
containers: | |
- name: couchdb | |
image: kocolosk/couchdb:2.0-prod | |
imagePullPolicy: Always | |
env: | |
- name: POD_IP | |
valueFrom: | |
fieldRef: | |
fieldPath: status.podIP | |
- name: ERL_FLAGS | |
value: "-name couchdb@$(POD_IP) -setcookie monster" | |
ports: | |
- name: couchdb | |
containerPort: 5984 | |
- name: epmd | |
containerPort: 4369 | |
- containerPort: 9100 | |
volumeMounts: | |
- name: database-storage | |
mountPath: /var/lib/couchdb | |
- name: mem3-kube | |
image: kocolosk/mem3-kube | |
imagePullPolicy: Always | |
volumes: | |
- name: database-storage | |
emptyDir: {} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
FROM debian:jessie | |
MAINTAINER Adam Kocoloski [email protected] | |
RUN groupadd -r couchdb && useradd -d /opt/couchdb -g couchdb couchdb | |
# Download dependencies | |
RUN apt-get update -y -qq && apt-get install -y --no-install-recommends \ | |
python python-pip \ | |
&& pip install requests | |
COPY mem3_kube.py /opt/mem3_kube/ | |
USER couchdb | |
WORKDIR /opt/mem3_kube | |
CMD ["mem3_kube.py"] | |
ENTRYPOINT ["/usr/bin/python"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This file just grabs the list of pod IP addresses for the service | |
# called "couchdb" and feeds those as `couchdb@PODIP` nodes to mem3. | |
# It's just a proof of concept; the proper solution is likely a module | |
# in mem3 itself. | |
import json | |
import os | |
import requests | |
import time | |
def discover_peers(): | |
f = open("/var/run/secrets/kubernetes.io/serviceaccount/token", 'r') | |
token = f.read() | |
uri = 'https://{0}:{1}/api/v1/namespaces/{2}/endpoints/{3}'.format( | |
os.environ.get("KUBERNETES_PORT_443_TCP_ADDR", "10.0.0.1"), | |
os.environ.get("KUBERNETES_PORT_443_TCP_PORT", "443"), | |
os.environ.get("POD_NAMESPACE", "default"), | |
os.environ.get("COUCHDB_SERVICE", "couchdb") | |
) | |
headers = { | |
'Authorization': "Bearer {0}".format(token) | |
} | |
resp = requests.get(uri, headers=headers, verify=False) | |
if resp.status_code == 200: | |
body = resp.json() | |
for key in ('addresses', 'notReadyAddresses'): | |
addresses = body['subsets'][0].get(key, []) | |
ips = [addr['ip'] for addr in addresses] | |
print key, ips | |
connect_the_dots(ips) | |
sleep_forever() | |
else: | |
print 'Error' | |
print resp.text | |
def connect_the_dots(ips): | |
for ip in ips: | |
uri = "http://localhost:5986/_nodes/couchdb@{0}".format(ip) | |
doc = {} | |
r = requests.put(uri, data = json.dumps(doc)) | |
print ip, r.status_code | |
def sleep_forever(): | |
while True: | |
time.sleep(5) | |
if __name__ == '__main__': | |
print os.environ | |
discover_peers() |
Adam, would you care to contribute this to the CouchDB docs somewhere? Or perhaps in apache/couchdb-pkg ? We can move you from kocolosk/couchdb:2.0-prod to apache/couchdb:2.1 as well (image being published today.)
Did this ever get canonically published anywhere afterall?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I deploy the deployment to my k8s cluster (on IBM Bluemix k8s service), the pod failed to start. Seems the mem3_kube couldn't find the couchdb service, with following error:
Back-off restarting failed docker container Error syncing pod, skipping: failed to "StartContainer" for "mem3-kube" with CrashLoopBackOff: "Back-off 10s restarting failed container=mem3-kube pod=couchdb-2626500488-w0298_default(a3e0a5d3-3f42-11e7-908b-528df0c526ee)"
I updated the deployment_couchdb.yaml file to add a service definition, it seems resolved the issue, here is my code:
`
apiVersion: v1
kind: Service
metadata:
name: couchdb
labels:
run: couchdb
spec:
ports:
port: 5984
targetPort: 5984
protocol: TCP
selector:
run: couchdb
`