gistfile1.txt

RabbitMQ queue types.

As 3.8 release with new quorum queue type approaches, the RabbitMQ team
was thinking of introducing a new concept for RabbitMQ: queue types.

Currently queues have many different settings, configured with arguments and
policies. These settings allow you to configure queues to be used in different
scenarios and have different behaviours.
For example mirrored/unmirrored, lazy, queues with limits and TTLs.

Some of those settinga don't dork well in combination and some have specific
behaviours you have to be aware of when using.

Queue types supposed to help you handling this complexity. A queue of some type
will have its own settings, configuring specific behaviour and created for
specific use-cases.

A type cannot be changed for existing queue and may not have settings available
in a different type.
Queue types can be optimised for one use-case and be more efficient and easy to
manage then configuring all the settings in a "classic" queue.

Right now we work on the "quorum queue" type and since 3.8 there will be two
types: "quorum" and "classic".

We plan to work on the queue type abstraction more to make in extensible, maybe
even with plugins.

We have some ideas about possible queue types to add in future, for example a
streaming queue (a la NATS), or low-latency queues, but these are only ideas
and may not be implemented or useful.

We would like to ask the community what do you think of this idea and what other
types of queues you can think of?
Are there any scenarios where classic queues need too much configuration or
don't fit the use case? 
Are there any queue use-cases you know may benefit from having a new queue type?

Please share your thoughts and ideas.

This is what I would post:

What queue type should we build next?

RabbitMQ queues can be configured in many different ways via queue arguments and policies. These queue settings make queues behave in ways suitable for specific workloads.

For example, a lazy queue mirrored to all nodes with manual sync-ing is desirable in scenarios where availability and message persistence are important. This configuration trades-off performance for message persistence and consistency. Mirroring has a side-effect of higher memory usage as well as network & CPU pressure uncharacteristic to lazy queues, but expected for mirrored queues. Furthermore, lazy queues have slightly different behaviour on node startup, which may be surprising for some: https://www.rabbitmq.com/lazy-queues.html#caveats-limitations. If message TTL is added to this queue configuration, more pressure will be put on all components to delete messages that used significant resources to be persisted and replicated in the first place.

From the example above, it is easy to see how certain queue configurations may be counter-productive and make queues behave in unexpected ways. Some settings imply behaviours that users need to be aware of, such as lazy queues and node startup.

We believe that queue types can help users make safer decisions. Our goal is to be upfront about trade-offs while narrowing down the possible queue configurations. The end-result is more predictable & consistent queue behaviour that is measurable and dependable.

In RabbitMQ 3.8, we introduce the first queue type: the quorum queue. You can learn more about Quorum Queues here: http://next.rabbitmq.com/quorum-queues.html

We have other queue type ideas, but would like to hear your opinions before we invest more time in this.

What queue type should we build next? Please describe your use-case with an emphasis on why certain queue characteristics are important to you.