At Burt we're heavy into RabbitMQ, we use it as a series of tubes that transports almost all of the wonderful data around our platform. It's served us well, apart from a tiny, tiny detail: give it too much of the good stuff and it sort of gets drunk and won't have anymore. Legend has it that version 2.8 solves that problem, and as far as we can tell it's almost all true. Read on for pretty graphs.
[[MORE]]
Our whole platform runs off of Amazon EC2, and for this test set we set up RabbitMQ clusters of 1, 2 and 3 c1.xlarge instances.
Each cluster gets 24 durable queues, we basically use RabbitMQ for transport, and to get the maximum throughput you want to have many queues, since each queue is essentially single threaded.
We loaded the queues by parsing about 2.3 gigs worth of messages from old production logs. The messages were combined and sent in batches of ten (another good technique for getting higher throughput), and routed explicitly to one of then 24 queues based on a property o