gistfile1.md

Discoverd Leader Election API

Notes:

• cannot be used for multiple leaders/quorum
• races between leader updates in servers/clients should be handled by message queuing and/or retry

Server Patterns

Many uncoordinated active servers (no leaders)

err := discoverd.Register("www", ":9099")
listenAndServe() // i.e. net.Listen("tcp", ":9099") 

Many standby servers, only one active server

standbyCh, err := discoverd.RegisterAndStandby("sampi", ":9099", nil)
<-standbyCh
listenAndServe()

Many active servers, one leader server, all servers ready to become leader

standbyCh, err := discoverd.RegisterAndStandby("mysql", ":9099", nil)
go func() {
	<-standbyCh
	upgradeToLeader()
}()
listenAndServe()

Client Patterns

Clients always wanting the one leader server

set, err := discoverd.ServiceSet("sampi")
for {
	leader := <-set.Leader() // receives current leader and any new future leaders
	go updateConnection(leader.Addr)
}

Clients with persistent connection pool connected to all active servers

set, err := discoverd.ServiceSet("app")
for update := range set.Updates() {
	if update.Online {
		connectionPool.Add(update.Addr)
	} else {
		connectionPool.Remove(update.Addr)
	}
}

Clients using non-persistent connections to random active server

rand.Seed(time.Now().UnixNano())
services, err := discoverd.Services("app", timeout)
conn, err := net.Dial("tcp", services[rand.Intn(len(services))].Addr)

Clients using non-persistent connections to leader server

services, err := discoverd.Services("app", timeout)
conn, err := net.Dial("tcp", services[0].Addr) // index 0 is always current leader at the time

Client/Server Hybrid Patterns

Many active servers, one leader server, all servers connect to leader and are ready to become leader

set, err := discoverd.RegisterWithSet("cluster", ":9099", nil)
go func() {
	for {
		leader := <-set.Leader() // receives current leader and any new future leaders
		if leader != nil {
			go updateConnection(leader.Addr)
		} else {
			upgradeToLeader()
		}
	}
}()
listenAndServe()

discoverd.RegisterWithSet can also be used similarly in order to handle "downgrading" from leader.

API Implementation Notes

set.Leader()

Lazily creates a channel to return and sends current leader on it, unless channel exists in which case it just returns existing channel and any changes of leader will continue to be sent to it. This means, once you call set.Leader() you have to always be receiving on it until you call set.Close()

discoverd.RegisterAndStandby()

The channel this returns will receive one service when the registered service is elected leader. It assumes it will only become leader once, so after that send, it won't attempt to send again.

discoverd.RegisterWithSet()

This returns a set from discoverd.ServiceSet() but excluding the service registered. This ensures it doesn't connect to itself. However, the set.Leader() will still fire when this service becomes leader, but since it wasn't in the set, the value will be nil, which is how it will know when it is elected leader.