Last active
October 19, 2015 04:09
-
-
Save danehans/d8e3e884158dcf334b3e to your computer and use it in GitHub Desktop.
netplugin magnum
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # my setup is 2 nodes, a kube master running kube-api/schedule/controller-mgr/docker/etcd | |
| # and a minion running docker/kubelet/proxy | |
| # Note: I was able to successfully instantiate pods/services prior to netplugin changes | |
| #kubelet now starts with netplugin | |
| # systemctl status kubelet.service -l | |
| ● kubelet.service - Kubernetes Kubelet Server | |
| Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled) | |
| Active: active (running) since Sun 2015-10-18 20:14:45 UTC; 23min ago | |
| Docs: https://github.com/GoogleCloudPlatform/kubernetes | |
| Main PID: 5331 (kubelet) | |
| CGroup: /system.slice/kubelet.service | |
| ├─5331 /usr/bin/kubelet --logtostderr=true --v=0 --api_servers=http://10.0.0.3:8080 --address=0.0.0.0 --allow_privileged=true --network_plugin=k8contivnet | |
| └─5344 journalctl -f | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.636117 5331 factory.go:226] System is using systemd | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.636823 5331 factory.go:234] Registering Docker factory | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.637444 5331 factory.go:89] Registering Raw factory | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.678261 5331 kubelet.go:806] Node k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky was previously registered | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.749981 5331 manager.go:946] Started watching for new ooms in manager | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.751333 5331 oomparser.go:183] oomparser using systemd | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.754012 5331 manager.go:243] Starting recovery of all containers | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.762807 5331 manager.go:248] Recovery completed | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.766108 5331 status_manager.go:76] Starting to sync pod status with apiserver | |
| Oct 18 20:14:46 k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky kubelet[5331]: I1018 20:14:46.766412 5331 kubelet.go:1725] Starting kubelet main sync loop. | |
| # I created a simple default json for netplugin: | |
| # cat default.json | |
| { | |
| "Hosts" : [{ | |
| "Name" : "k8-lixyr4eict-0-rs23viw2j2b2-kube-minion-hy6fbdyywiky", | |
| "VtepIp" : "10.0.0.6" | |
| }], | |
| "Tenants" : [ { | |
| "Name" : "tenant-one", | |
| "DefaultNetType" : "vxlan", | |
| "SubnetPool" : "11.1.0.0/16", | |
| "AllocSubnetLen" : 24, | |
| "Vxlans" : "10001-14000", | |
| "Networks" : [ { | |
| "Name" : "demo1", | |
| "Endpoints" : [ { | |
| "Container" : "nginx" | |
| } ] | |
| } ] | |
| } ] | |
| } | |
| # I follow the kubernetes.md directions, but am unable | |
| # to instantiate the default network using default.json (above). | |
| # I get the following error: | |
| # netdcli -cfg /etc/sysconfig/contiv/default.json | |
| FATA[0000] error executing the command. Opts: {help:false debug:false cfgDesired:true cfgAdditions:false cfgDeletions:false cfgHostBindings:false oper:{val:} construct:{val:} stateStore:etcd storeURL: netmasterURL:localhost:9999 tenant:default netID: pktTag:auto pktTagType:vlan subnetCidr: ipAddr:auto contName: attachUUID: subnetIP: subnetLen:0 allocSubnetLen:24 defaultGw: idStr:/etc/sysconfig/contiv/default.json vlans: vxlans: homingHost: vtepIP: intfName:}. Error: error processing cfg. Error: Post http://localhost:9999/desired-config: dial tcp [::1]:9999: getsockopt: connection refused [main.executeJSONCfg netdcli.go 598] | |
| # I try to start netmaster (Note: netmaster is not documented in kubernetes.md) on my kube master, | |
| # but I get the following error: | |
| # netmaster | |
| INFO[0000] Got host addr: 10.0.0.5 | |
| INFO[0000] Registering service key: /contiv.io/service/netmaster/10.0.0.5:9999, value: {ServiceName:netmaster HostAddr:10.0.0.5 Port:9999} | |
| ERRO[0000] Error setting key /contiv.io/service/netmaster/10.0.0.5:9999, Err: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0] | |
| FATA[0000] Error registering service. Err: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0] | |
| # It apears that I may have to supply the etcd store-url to netmaster. | |
| # I supply the etcd url (same url used by kube-apiserevr service). | |
| # However, I get the following error: | |
| # netmaster -debug -store-url="http://127.0.0.1:2379" | |
| DEBU[0000] Results for (http://localhost:2379/v2/stats/self): &{Name:10.0.0.5} | |
| DEBU[0000] Results for (http://localhost:2379/v2/members): &{Members:[{Name:10.0.0.5 ClientURLs:[http://10.0.0.5:2379]}]} | |
| INFO[0000] Got host addr: 10.0.0.5 | |
| INFO[0000] Registering service key: /contiv.io/service/netmaster/10.0.0.5:9999, value: {ServiceName:netmaster HostAddr:10.0.0.5 Port:9999} | |
| ERRO[0000] Error setting key /contiv.io/service/netmaster/10.0.0.5:9999, Err: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0] | |
| FATA[0000] Error registering service. Err: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0] | |
| # To etcd functionality, I use the same url to set a test key from the kube master: | |
| # curl -L http://127.0.0.1:2379/v2/keys/mykey -XPUT -d value="this is awesome" | |
| {"action":"set","node":{"key":"/mykey","value":"this is awesome","modifiedIndex":464,"createdIndex":464}} | |
| # I then check the key | |
| # curl -L http://127.0.0.1:2379/v2/keys/mykey | |
| {"action":"get","node":{"key":"/mykey","value":"this is awesome","modifiedIndex":464,"createdIndex":464}} | |
| # I can also set and read the same k/v pair that netmaster is trying to use: | |
| # curl -L http://127.0.0.1:2379/v2/keys/contiv.io/service/netmaster/10.0.0.5:9999 -XPUT -d value="{ServiceName:netmaster HostAddr:10.0.0.5 Port:9999}" | |
| {"action":"set","node":{"key":"/contiv.io/service/netmaster/10.0.0.5:9999","value":"{ServiceName:netmaster HostAddr:10.0.0.5 Port:9999}","modifiedIndex":481,"createdIndex":481}} | |
| # curl -L http://127.0.0.1:2379/v2/keys/contiv.io/service/netmaster/10.0.0.5:9999 | |
| {"action":"get","node":{"key":"/contiv.io/service/netmaster/10.0.0.5:9999","value":"{ServiceName:netmaster HostAddr:10.0.0.5 Port:9999}","modifiedIndex":481,"createdIndex":481}} | |
| # Even after seeting this k/v pair, netmaster does not start: | |
| # netmaster -debug -store-url="http://127.0.0.1:2379" | |
| DEBU[0000] Results for (http://localhost:2379/v2/stats/self): &{Name:10.0.0.5} | |
| DEBU[0000] Results for (http://localhost:2379/v2/members): &{Members:[{Name:10.0.0.5 ClientURLs:[http://10.0.0.5:2379]}]} | |
| INFO[0000] Got host addr: 10.0.0.5 | |
| INFO[0000] Registering service key: /contiv.io/service/netmaster/10.0.0.5:9999, value: {ServiceName:netmaster HostAddr:10.0.0.5 Port:9999} | |
| ERRO[0000] Error setting key /contiv.io/service/netmaster/10.0.0.5:9999, Err: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0] | |
| FATA[0000] Error registering service. Err: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0] | |
| # This is the version of etcd that is being used: | |
| # rpm -qa | grep etcd | |
| etcd-2.0.10-1.fc21.x86_64 | |
| # I am using netplugin-v0.0.0-10-07-2015.18-32-49.UTC.tar.bz2 | |
| # kubectl version | |
| Client Version: version.Info{Major:"1", Minor:"0", GitVersion:"v1.0.4", GitCommit:"65d28d5fd12345592405714c81cd03b9c41d41d9", GitTreeState:"clean"} | |
| Server Version: version.Info{Major:"1", Minor:"0", GitVersion:"v1.0.4", GitCommit:"65d28d5fd12345592405714c81cd03b9c41d41d9", GitTreeState:"clean"} | |
| # rpm -qa | grep kube | |
| kubernetes-1.0.0-0.9.git2d88675.fc21.x86_64 | |
| kubernetes-client-1.0.0-0.9.git2d88675.fc21.x86_64 | |
| kubernetes-master-1.0.0-0.9.git2d88675.fc21.x86_64 | |
| kubernetes-node-1.0.0-0.9.git2d88675.fc21.x86_64 | |
| # docker version | |
| Client: | |
| Version: 1.8.1.fc21 | |
| API version: 1.20 | |
| Package Version: docker-io-1.8.1-2.git32b8b25.fc21.x86_64 | |
| Go version: go1.5.1 | |
| Git commit: 32b8b25/1.8.1 | |
| Built: | |
| OS/Arch: linux/amd64 | |
| Server: | |
| Version: 1.8.1.fc21 | |
| API version: 1.20 | |
| Package Version: | |
| Go version: go1.5.1 | |
| Git commit: 32b8b25/1.8.1 | |
| Built: | |
| OS/Arch: linux/amd64 | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment