Skip to content

Instantly share code, notes, and snippets.

@walterheck
Created September 3, 2013 18:36
Show Gist options
  • Save walterheck/6427822 to your computer and use it in GitHub Desktop.
Save walterheck/6427822 to your computer and use it in GitHub Desktop.
Sep 3 12:13:00 vagrant-debian-squeeze mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Sep 3 12:13:00 vagrant-debian-squeeze mysqld_safe: WSREP: Running position recovery with --log_error=/tmp/tmp.Sm9NLXSZDi --pid-file=/var/lib/mysql/vagrant-debian-squeeze-recover.pid
Sep 3 12:13:02 vagrant-debian-squeeze mysqld_safe: WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: wsrep_start_position var submitted: '00000000-0000-0000-0000-000000000000:-1'
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Read nil XID from storage engines, skipping position init
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera/libgalera_smm.so'
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: wsrep_load(): Galera 23.2.6-squeeze(r) by Codership Oy <[email protected]> loaded succesfully.
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Reusing existing '/var/lib/mysql//galera.cache'.
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Passing config to GCS: base_host = 192.168.178.202; base_port = 4567; cert.log_conflicts = no; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_siz = 512M; gcache.page_size = 128M; gcache.size = 512M; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 16; gcs.fc_master_slave = NO; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = NO; replicator.causal_read_timeout = PT30S; replicator.commit_order = 3
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: wsrep_sst_grab()
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Start replication
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: protonet asio version 0
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: backend: asio
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: GMCast version 0
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: (1713e3be-14bc-11e3-80df-5ffcff6418f7, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: (1713e3be-14bc-11e3-80df-5ffcff6418f7, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: EVS version 0
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: PC version 0
Sep 3 12:13:02 vagrant-debian-squeeze mysqld: 130903 12:13:02 [Note] WSREP: gcomm: connecting to group 'olindata-galera', peer '192.168.178.201:'
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: declaring 0173434e-14c7-11e3-bb95-ae5977df8078 stable
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Node 0173434e-14c7-11e3-bb95-ae5977df8078 state prim
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: view(view_id(PRIM,0173434e-14c7-11e3-bb95-ae5977df8078,2) memb {
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #0110173434e-14c7-11e3-bb95-ae5977df8078,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #0111713e3be-14bc-11e3-80df-5ffcff6418f7,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: } joined {
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: } left {
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: } partitioned {
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: })
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: gcomm: connected
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Opened channel 'olindata-galera'
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Waiting for SST to complete.
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: STATE EXCHANGE: sent state msg: 3b755a58-14c7-11e3-89df-faabff3faf0c
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: STATE EXCHANGE: got state msg: 3b755a58-14c7-11e3-89df-faabff3faf0c from 0 (node1)
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: STATE EXCHANGE: got state msg: 3b755a58-14c7-11e3-89df-faabff3faf0c from 1 (node2)
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Quorum results:
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011version = 2,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011component = PRIMARY,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011conf_id = 1,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011members = 1/2 (joined/total),
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011act_id = 0,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011last_appl. = -1,
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011protocols = 0/4/2 (gcs/repl/appl),
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011group UUID = 4b9393cf-14bd-11e3-b9f8-96759e90db98
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Flow-control interval: [23, 23]
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 0)
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: State transfer required:
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011Group state: 4b9393cf-14bd-11e3-b9f8-96759e90db98:0
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Note] WSREP: New cluster view: global state: 4b9393cf-14bd-11e3-b9f8-96759e90db98:0, view# 2: Primary, number of nodes: 2, my index: 1, protocol version 2
Sep 3 12:13:03 vagrant-debian-squeeze mysqld: 130903 12:13:03 [Warning] WSREP: Gap in state sequence. Need state transfer.
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: 130903 12:13:05 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '192.168.178.202' --auth 'root:root' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '11855''
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: WSREP_SST: [INFO] Streaming with tar (20130903 12:13:05.512)
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: WSREP_SST: [INFO] Note: Advanced xtrabackup features - encryption,compression etc. not available with tar. (20130903 12:13:05.513)
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: WSREP_SST: [INFO] However, xbstream requires manual cleanup of data directory before SST - lp:1193240. (20130903 12:13:05.515)
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: nc: invalid option -- 'd'
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: nc -h for help
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: tar: This does not look like a tar archive
Sep 3 12:13:05 vagrant-debian-squeeze mysqld: tar: Exiting with failure status due to previous errors
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Note] WSREP: Prepared SST request: xtrabackup|192.168.178.202:4444/xtrabackup_sst
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Note] WSREP: Assign initial position for certification: 0, protocol version: 2
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (4b9393cf-14bd-11e3-b9f8-96759e90db98): 1 (Operation not permitted)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():436. IST will be unavailable.
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Note] WSREP: Node 1 (node2) requested state transfer from '*any*'. Selected 0 (node1)(SYNCED) as donor.
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 0)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [Note] WSREP: Requesting state transfer: success, donor: 0
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: WSREP_SST: [ERROR] Error while getting data from donor node: exit codes: 1 2 (20130903 12:13:15.871)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: WSREP_SST: [ERROR] Data directory /var/lib/mysql/ needs to be empty for SST: Manual intervention required in that case (20130903 12:13:15.873)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: WSREP_SST: [ERROR] Cleanup after exit with status:32 (20130903 12:13:15.875)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: WSREP_SST: [ERROR] Killing nc pid (20130903 12:13:15.883)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: WSREP_SST: [INFO] Removing the sst_in_progress file (20130903 12:13:15.885)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'joiner' --address '192.168.178.202' --auth 'root:root' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '11855': 32 (Broken pipe)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [ERROR] WSREP: SST failed: 32 (Broken pipe)
Sep 3 12:13:15 vagrant-debian-squeeze mysqld: 130903 12:13:15 [ERROR] Aborting
Sep 3 12:13:15 vagrant-debian-squeeze mysqld:
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Warning] WSREP: 0 (node1): State transfer to 1 (node2) failed: -1 (Operation not permitted)
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [ERROR] WSREP: gcs/src/gcs_group.c:gcs_group_handle_join_msg():719: Will never receive state. Need to abort.
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: gcomm: terminating thread
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: gcomm: joining thread
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: gcomm: closing backend
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: Closing send monitor...
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: Closed send monitor.
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: gcomm: terminating thread
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: gcomm: joining thread
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [Note] WSREP: gcomm: closing backend
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: 130903 12:13:17 [ERROR] WSREP: failed to close gcomm backend connection: 131: Forbidden state transition: LEAVING -> LEAVING (FATAL)
Sep 3 12:13:17 vagrant-debian-squeeze mysqld: #011 at gcomm/src/evs_proto.cpp:shift_to():2185
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: 130903 12:13:18 [ERROR] WSREP: exception caused by message: evs::msg{version=0,type=5,user_type=255,order=1,seq=34,seq_range=-1,aru_seq=34,flags=4,source=0173434e-14c7-11e3-bb95-ae5977df8078,source_view_id=view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2),range_uuid=00000000-0000-0000-0000-000000000000,range=[-1,-1],fifo_seq=169,node_list=(#0110173434e-14c7-11e3-bb95-ae5977df8078,node: {operational=1,suspected=0,leave_seq=-1,view_id=view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2),safe_seq=34,im_range=[36,35],}
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0111713e3be-14bc-11e3-80df-5ffcff6418f7,node: {operational=0,suspected=0,leave_seq=34,view_id=view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2),safe_seq=34,im_range=[35,34],}
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: )
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: }
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: state after handling message: evs::proto(evs::proto(1713e3be-14bc-11e3-80df-5ffcff6418f7, LEAVING, view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2)), LEAVING) {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: current_view=view(view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2) memb {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0110173434e-14c7-11e3-bb95-ae5977df8078,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0111713e3be-14bc-11e3-80df-5ffcff6418f7,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: } joined {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: } left {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: } partitioned {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: }),
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: input_map=evs::input_map: {aru_seq=34,safe_seq=34,node_index=node: {idx=0,range=[36,35],safe_seq=34} node: {idx=1,range=[35,34],safe_seq=34} },
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: fifo_seq=60,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: last_sent=34,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: known={
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0110173434e-14c7-11e3-bb95-ae5977df8078,evs::node{operational=1,suspected=0,installed=1,fifo_seq=169,}
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0111713e3be-14bc-11e3-80df-5ffcff6418f7,evs::node{operational=1,suspected=0,installed=1,fifo_seq=-1,leave_message=
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: evs::msg{version=0,type=6,user_type=255,order=1,seq=34,seq_range=-1,aru_seq=34,flags=0,source=1713e3be-14bc-11e3-80df-5ffcff6418f7,source_view_id=view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2),range_uuid=00000000-0000-0000-0000-000000000000,range=[-1,-1],fifo_seq=59,node_list=()
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: },
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: }
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: }
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: }130903 12:13:18 [ERROR] WSREP: failed to close gcomm backend connection: 131: evs::proto(evs::proto(1713e3be-14bc-11e3-80df-5ffcff6418f7, LEAVING, view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2)), LEAVING) {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: current_view=view(view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2) memb {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0110173434e-14c7-11e3-bb95-ae5977df8078,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0111713e3be-14bc-11e3-80df-5ffcff6418f7,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: } joined {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: } left {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: } partitioned {
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: }),
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: input_map=evs::input_map: {aru_seq=34,safe_seq=34,node_index=node: {idx=0,range=[36,35],safe_seq=34} node: {idx=1,range=[35,34],safe_seq=34} },
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: fifo_seq=60,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: last_sent=34,
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: known={
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0110173434e-14c7-11e3-bb95-ae5977df8078,evs::node{operational=1,suspected=0,installed=1,fifo_seq=169,}
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: #0111713e3be-14bc-11e3-80df-5ffcff6418f7,evs::node{operational=1,suspected=0,installed=1,fifo_seq=-1,leave_message=
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: evs::msg{version=0,type=6,user_type=255,order=1,seq=34,seq_range=-1,aru_seq=34,flags=0,source=1713e3be-14bc-11e3-80df-5ffcff6418f7,source_view_id=view_id(REG,0173434e-14c7-11e3-bb95-ae5977df8078,2),range_uuid=00000000-0000-0000-0000-000000000000,r
Sep 3 12:13:18 vagrant-debian-squeeze mysqld: 130903 12:13:18 [Note] WSREP: /usr/sbin/mysqld: Terminated.
Sep 3 12:13:18 vagrant-debian-squeeze mysqld_safe: mysqld from pid file /var/run/mysqld/mysqld.pid ended
Sep 3 12:13:30 vagrant-debian-squeeze /etc/init.d/mysql[12542]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Sep 3 12:13:30 vagrant-debian-squeeze /etc/init.d/mysql[12542]: #007/usr/bin/mysqladmin: connect to server at 'localhost' failed
Sep 3 12:13:30 vagrant-debian-squeeze /etc/init.d/mysql[12542]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)'
Sep 3 12:13:30 vagrant-debian-squeeze /etc/init.d/mysql[12542]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
Sep 3 12:13:30 vagrant-debian-squeeze /etc/init.d/mysql[12542]:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment