Using Force Remove to Change a nodes IP address.

##Using Force Replace to Change a nodes IP address. ###Objective Previous to Riak 1.2, a cluster node's IP address could be changed by running riak-admin reip on each cluster node. As of Riak 1.2, this method has been replaced with riak-admin cluster force-replace which is safer and does not require any downtime.

###Scenario

Riak is running in a cluster of five nodes.

[email protected] on node1.localdomain (192.168.17.11)(64.214.83.25)
[email protected] on node2.localdomain (192.168.17.12)(64.214.83.26)
[email protected] on node3.localdomain (192.168.17.13)(64.214.83.27)
[email protected] on node4.localdomain (192.168.17.14)(64.214.83.28)
[email protected] on node5.localdomain (192.168.17.15)(64.214.83.29)

The cluster is currently using the public IPs for communication.

The goal is to change the cluster to work over the private IP addresses with no downtime.

The Process tl;dr

This process can be accomplished in three steps, the details of which will be discussed below.

Down the Node
Reconfigure the node to Listen on the Private IP
Repeat

The Process

Down the Node

Stop riak on node1.localdomain if the node is still running in any way.

riak stop

node1> riak stop
Attempting to restart script through sudo -H -u riak
ok
node1>

Mark [email protected] down from node2.localdomain

riak-admin down [email protected]

```
node2> riak-admin down [email protected]
Attempting to restart script through sudo -H -u riak
Success: "[email protected]" marked as down
node2>
```
This will tell the cluster that this node is offline and ring-state transtions should be allowed, and can be run from any running cluster node.

Reconfigure the node to Listen on the Private IP

Reconfigure node1.localdomain to listen on the private IP addresses by doing the following:

Edit the vm.args file on the new node and set the -name argument as follows:
```
-name [email protected]
```
Change the IP addresses in app.config as appropriate, specifically pb_ip,http,https, and cluster_mgr settings. Verify that they are listening on the appropriate addresses. These are generally load-balancer or public facing IP addresses.
Rename the ring folder. The location of the ring folder is listed in the /etc/riak/app.config.

Start Riak on node1.localdomain.

riak start

node1> riak start
Attempting to restart script through sudo -H -u riak    
node1>

Join the node back into the cluster.

riak-admin cluster join [email protected]

node1> riak-admin cluster join [email protected]
Attempting to restart script through sudo -H -u riak
Success: staged join request for '[email protected]' to '[email protected]'
node1>

Use force-replace to change all ownership references from [email protected] to [email protected].

riak-admin cluster force-replace [email protected] [email protected]

```
node1> riak-admin cluster force-replace [email protected] [email protected]
Attempting to restart script through sudo -H -u riak
Success: staged forced replacement of '[email protected]' with '[email protected]'
node1>
```

Show the planned cluster changes.

riak-admin cluster plan

node1> riak-admin cluster plan
Attempting to restart script through sudo -H -u riak
=========================== Staged Changes ============================
Action         Nodes(s)
-----------------------------------------------------------------------
join           '[email protected]'
force-replace  '[email protected]' with '[email protected]'
-----------------------------------------------------------------------

WARNING: All of '[email protected]' replicas will be lost

NOTE: Applying these changes will result in 1 cluster transition

#######################################################################
                     After cluster transition 1/1
#######################################################################

============================= Membership ==============================
Status     Ring    Pending    Node
-----------------------------------------------------------------------
valid      20.3%      --      '[email protected]'
valid      20.3%      --      '[email protected]'
valid      20.3%      --      '[email protected]'
valid      20.3%      --      '[email protected]'
valid      18.8%      --      '[email protected]'
-----------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Partitions reassigned from cluster changes: 13
  13 reassigned from '[email protected]' to '[email protected]'

node1>

Commit the changes to the cluster.

riak-admin cluster commit

node1> riak-admin cluster commit
Attempting to restart script through sudo -H -u riak
Cluster changes committed
node1>

Check that everything connected and functioning as expected

riak-admin member-status

node1> riak-admin member-status
Attempting to restart script through sudo -H -u riak
============================= Membership ==============================
Status     Ring    Pending    Node
-----------------------------------------------------------------------
valid      20.3%      --      '[email protected]'
valid      20.3%      --      '[email protected]'
valid      20.3%      --      '[email protected]'
valid      20.3%      --      '[email protected]'
valid      18.8%      --      '[email protected]'
-----------------------------------------------------------------------
Valid:5 / Leaving:0 / Exiting:0 / Joining:0 / Down:0

Monitor riak-admin transfers to make sure that all hinted handoff is complete.
Delete the renamed ring folder to clean up once everything is done.

Repeat

Repeat for the remaining nodes. Use [email protected] for further cluster join commands in Replace:4