turn off pacemaker and bootstrap galera manually

gistfile1.txt

--- 1. prevent pacemaker from managing galera

# pcs resource unmanage galera

--- 2. make sure ***mysql is not running on any node***.  To bootstrap, mysql has to be completely shut down:

# ps -ef | grep mysql
< nothing, on all three controllers >

--- 3. Choose the node you want to bootstrap and run mysqld_safe with --wsrep-new-cluster:

# /usr/bin/mysqld_safe --defaults-file=/etc/my.cnf --pid-file=/var/run/mysql/mysqld.pid --socket=/var/lib/mysql/mysql.sock --datadir=/var/lib/mysql --log-error=/var/log/mysqld.log --user=mysql --open-files-limit=16384 --wsrep-cluster-address=gcomm://overcloud-controller-0.localdomain,overcloud-controller-1.localdomain,overcloud-controller-2.localdomain  --wsrep-new-cluster &


--- 4. Go to the two other nodes and start them *without* wsrep-new-cluster:

# /usr/bin/mysqld_safe --defaults-file=/etc/my.cnf --pid-file=/var/run/mysql/mysqld.pid --socket=/var/lib/mysql/mysql.sock --datadir=/var/lib/mysql --log-error=/var/log/mysqld.log --user=mysql --open-files-limit=16384 --wsrep-cluster-address=gcomm://overcloud-controller-0.localdomain,overcloud-controller-1.localdomain,overcloud-controller-2.localdomain  &

The nodes will need to rsync over the 124G file which will take over five minutes.  the mysqld.log files on all three nodes will show current state.  A node that is in JOINER is receiving an SST.  A node that is in DONOR is sending an SST.  a node that is SYNCED is ready to go.  You want all three mysqld.log files to report SYNCED as the most recent state, and then the logfiles should be still.

You can run clustercheck on all three nodes to make sure they are ready to receive connections:

[root@overcloud-controller-0 /]# clustercheck
HTTP/1.1 200 OK
Content-Type: text/plain
Connection: close
Content-Length: 32

Galera cluster node is synced.


at this point you can pcs resource manage galera.  it should show the cluster as three masters all up.