-
-
Save smreed/e93fd74765caef99266a to your computer and use it in GitHub Desktop.
This gist represents all the manifests you'll need to run a redundant and fault-tolerant NSQ cluster on Kubernetes. NSQ Admin's http interface is available on port 14171 on any node. There's an issue with NSQ Admin if you're running more than one lookupd service, so I recommend running just one lookupd until I figure it out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqadmin-http", | |
"kind": "Service", | |
"apiVersion": "v1beta1", | |
"containerPort": 4171, | |
"port": 14171, | |
"protocol": "TCP", | |
"selector": { "name": "nsqadmin" }, | |
"createExternalLoadBalancer": true | |
} | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqadminController", | |
"kind": "ReplicationController", | |
"apiVersion": "v1beta1", | |
"desiredState": { | |
"replicas": 1, | |
"replicaSelector": {"name": "nsqadmin"}, | |
"podTemplate": { | |
"desiredState": { | |
"manifest": { | |
"version": "v1beta1", | |
"id": "nsqadmin", | |
"containers": [{ | |
"name": "nsqadmin", | |
"image": "smreed/nsqadmin", | |
"ports": [{"containerPort": 4171}], | |
}] | |
} | |
}, | |
"labels": {"name": "nsqadmin"} | |
}}, | |
"labels": {"name": "nsqadmin"} | |
} | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqd-http", | |
"kind": "Service", | |
"apiVersion": "v1beta1", | |
"port": 14151, | |
"containerPort": 4151, | |
"protocol": "TCP", | |
"selector": { "name": "nsqd" } | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqd-tcp", | |
"kind": "Service", | |
"apiVersion": "v1beta1", | |
"port": 14150, | |
"containerPort": 4150, | |
"protocol": "TCP", | |
"selector": { "name": "nsqd" } | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqdController", | |
"kind": "ReplicationController", | |
"apiVersion": "v1beta1", | |
"desiredState": { | |
"replicas": 5, | |
"replicaSelector": {"name": "nsqd"}, | |
"podTemplate": { | |
"desiredState": { | |
"manifest": { | |
"version": "v1beta1", | |
"id": "nsqd", | |
"volumes": {["name": "nsqdpersistence"]}, | |
"containers": [{ | |
"name": "nsqd", | |
"image": "smreed/nsqd", | |
"volumes": {["name":"nsqdpersistence","mountPath":"/data"]}, | |
"ports": [ | |
{"containerPort": 4150}, | |
{"containerPort": 4151} | |
] | |
}] | |
} | |
}, | |
"labels": {"name": "nsqd"} | |
}}, | |
"labels": {"name": "nsqd"} | |
} | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqlookupd-http", | |
"kind": "Service", | |
"apiVersion": "v1beta1", | |
"port": 14161, | |
"containerPort": 4161, | |
"protocol": "TCP", | |
"selector": { "name": "nsqlookupd" } | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqlookupd-tcp", | |
"kind": "Service", | |
"apiVersion": "v1beta1", | |
"port": 14160, | |
"containerPort": 4160, | |
"protocol": "TCP", | |
"selector": { "name": "nsqlookupd" } | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"id": "nsqlookupdController", | |
"kind": "ReplicationController", | |
"apiVersion": "v1beta1", | |
"desiredState": { | |
"replicas": 1, | |
"replicaSelector": {"name": "nsqlookupd"}, | |
"podTemplate": { | |
"desiredState": { | |
"manifest": { | |
"version": "v1beta1", | |
"id": "nsqlookupdController", | |
"containers": [{ | |
"name": "nsqlookupd", | |
"image": "nsqio/nsqlookupd", | |
"ports": [ | |
{"containerPort": 4160}, | |
{"containerPort": 4161} | |
] | |
}] | |
} | |
}, | |
"labels": {"name": "nsqlookupd"} | |
}}, | |
"labels": {"name": "nsqlookupd"} | |
} |
I don't see how this could work in production... The volume are not persistent, so they will be destroyed with the pods. If you setup a persistent volume, all the instances will read/write in the same fs, leading to file corruption.
Also, since nsqd instances are LB by the service, it will be seen as one nsqd only by the clients. There's no coordination / replication between nsqd instances, so the consumers will probably see nothing until the LB arrives on the instance with a message, which seems pretty inefficient.
I think the only solution is to have a service and a persistent volume PER instance, which is a lot more configuration, and it's not scalable.
What do you think?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@mindscratch I just built an image of nsqd based on @smreed one. It solves the issue by broadcasting the IP instead of the hostname.
use asibiril/nsqd instead of smreed/nsqd