Skip to content

Instantly share code, notes, and snippets.

@MarcialRosales

MarcialRosales/README.md

Last active September 20, 2019 16:11
Show Gist options
  • Save MarcialRosales/21dbe7700fcee1db6f110ba7cd0e57de to your computer and use it in GitHub Desktop.
Save MarcialRosales/21dbe7700fcee1db6f110ba7cd0e57de to your computer and use it in GitHub Desktop.
Download ZIP
RabbitMQ Queue Distribution Strategies
Raw
README.md

RabbitMQ Queue Distribution Strategies

Queues can be distributed between nodes by either

  • expliciting declaring the queues in the node where we want it to go,
  • or delegating to RabbitMQ to evenly distributed the queue for us

These two options are fully explained in the rabbitmq docs.

However it is important to understand that when nodes are restarted, say during a rolling upgrade, we may end up with an unbalanced cluster. RabbitMQ will NOT automatically move queues in an attempt to obtain a balanced cluster.

Say for instance that we have 3 node cluster (nodeA, nodeB, nodeC) with all 3 nodes up and running. Applications come along and declare 9 non-durable queues. We have configured RabbitMQ with queue_master_locator equal to min-masters therefore we end up with 3 queues on each node: nodeA(3), nodeB(3), nodeC(3). We shut down nodeC for maintenance. Applications reconnect to nodeA or nodeB and redeclare the queues. Now we have nodeA(4), nodeB(5). NodeC is brought back and now we have the following unbalanced distribution: nodeA(4), nodeB(5), nodeC(0).

So far, the only way to move a queue from one node to another is to use a technique implemented by an script called rebalance-queue-masters provided by the RabbitMQ team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment