RabbitMQ in Depth

What RabbitMQ is, in one minute

RabbitMQ is a message broker: services hand it messages, and it delivers them to the services that should process them. It differs from Kafka in two ways that shape every design:

• The broker routes. Publishers send to an exchange, and the exchange decides which queues get a copy, based on rules called bindings. Publishers never need to know who consumes.
• Messages are deleted once handled. A consumer acknowledges a message and it is gone. That makes RabbitMQ a natural fit for work queues ("process this job once"), and a poor fit for replaying history, unless you use its streams.

Exchanges and routing

Exchange type	Routes by	Example
Direct	exact routing key	key payments → the payments queue
Fanout	nothing: every bound queue gets a copy	broadcast a cache-invalidation event
Topic	key patterns: * matches one word, # matches zero or more	order.* gets order.created and order.paid
Headers	message header values	rare; route on several attributes

One trap to know early: an exchange does not store messages. If no queue matches, the message is silently dropped, unless the publisher sets mandatory=true (the message comes back to it) or the exchange has an alternate exchange to catch it.

Choosing a queue type

• Classic queues: a queue lives on one node. Fast and simple, but if that node is lost, so are its messages. Classic queue mirroring was removed in RabbitMQ 4.0.
• Quorum queues: replicated across several nodes (three by default, capped at the cluster size) using the Raft consensus algorithm. A message is confirmed once a majority of members have it, so a 3-member queue survives the loss of one node. This is the choice for data you care about.
• Streams: an append-only log, like a Kafka partition. Reading does not delete, so many consumers can read the same messages and replay from any point.

The settings that matter

Setting	Default	What it decides
Prefetch (basic.qos)	0 (no limit)	how many unacknowledged messages a consumer may hold
Publisher confirms	off	whether publishers learn that the broker safely has a message
mandatory flag	false	whether unroutable messages are returned or dropped
Quorum delivery-limit	20 (since 4.0)	how many failed deliveries a message may have before it is dropped or dead-lettered
vm_memory_high_watermark	0.6 (60% of RAM)	when the broker blocks publishers to protect itself

Not losing messages: confirms and acks

Safety needs both ends:

1. Publisher confirms: the broker tells the publisher when it has safely stored a message (for a quorum queue, on a majority). Without confirms, a publisher has no way to know a message was lost in a crash or a dropped connection.
2. Manual consumer acks: the consumer acknowledges after processing. If it crashes first, the message is redelivered to another consumer. With automatic acks, RabbitMQ forgets the message the moment it is sent.

Both mean the same thing as everywhere else: delivery is at least once, so consumers must be idempotent.

Prefetch: the setting that decides fairness

With the default prefetch of 0 (no limit), RabbitMQ pushes waiting messages to consumers as fast as it can. The first consumer to connect can receive almost the whole backlog, holding it in memory while new consumers sit idle. Setting a prefetch caps how many unacknowledged messages each consumer holds:

• Too low (1): each consumer waits for a round trip after every message. Safe and fair, but slow.
• Too high or unlimited: unfair distribution and memory growth in consumers.
• In between, tens to a few hundred is common; measure with your message sizes and processing times. (Quorum queues cap prefetch at 2,000 regardless.)

Dead-lettering

A queue can name a dead-letter exchange (DLX), usually set by policy. A message is dead-lettered when:

• a consumer rejects it with basic.reject or basic.nack and requeue=false;
• it expires because of a message TTL;
• it is dropped because the queue hit a length limit;
• it failed delivery more times than a quorum queue's delivery limit.

Without a DLX, these messages are simply gone. With one, they land in a queue where people (or a retry job) can look at them.

Ordering

A single queue is first-in, first-out. Ordering breaks as soon as there are several consumers (they process in parallel) or a message is requeued (it goes back near the front and may be handled after later ones). If order matters, use Single Active Consumer on the queue, so only one consumer receives messages at a time and another takes over if it fails, or route related messages to their own queue.

Going deeper: quorum queue trade-offs

Quorum queues need a majority of members online to accept writes: a 3-member queue tolerates one failure, a 5-member queue two. Members should be an odd number for the same reason as any Raft cluster.

They are always durable (no transient quorum queues), they keep a log on disk, and every message is replicated, so throughput per queue is lower than a classic queue on one node. In exchange, a node failure loses nothing that was confirmed. The built-in delivery limit (20 by default since 4.0) is a safety net against poison messages looping forever, which classic queues lack. Know what counts: since RabbitMQ 4.3, only genuine failures (a consumer crash or lost connection, basic.reject) count toward the limit. A basic.nack with requeue no longer does, so a consumer that keeps nacking the same message can still loop.

Going deeper: flow control and long queues

When a node's memory use crosses the high watermark (60% of RAM by default), or free disk falls below its limit, RabbitMQ blocks publishing connections until the pressure eases. Publishers simply stall, which is better than the broker crashing, but it surprises teams who did not plan for it.

The usual cause is a queue growing without bound because consumers are slower than producers, or consumers holding thousands of unacknowledged messages. RabbitMQ is happiest with short queues. Set a length limit (x-max-length) with an overflow behavior you chose on purpose, alert on queue depth and on the age of the oldest message, and scale consumers before the alarm does it for you.

How it fails

• Silent drops: unroutable messages with mandatory=false and no alternate exchange.
• Poison loops: a consumer that always requeues a bad message. Reject it with requeue=false and a DLX. Do not count on the delivery limit to stop a basic.nack loop: since 4.3 those returns are not counted.
• Unfair work and memory growth: unlimited prefetch.
• Blocked publishers: memory or disk alarms caused by long queues.
• Lost messages on node failure: classic queues for important data.

RabbitMQ or Kafka?

Choose RabbitMQ for work queues and flexible routing: each job is done once, routing rules change often, per-message acks and dead-lettering matter. Choose Kafka when many independent consumers need the same events, when you need to replay history, or for very high throughput with per-key ordering. RabbitMQ streams and Kafka share groups have narrowed the gap, but the core models still differ.

What to say in a design review

• "Payments use a quorum queue, publisher confirms and manual acks, so a confirmed message survives a node loss and is processed at least once."
• "Consumers have a prefetch of 50 and idempotent handlers."
• "Every queue has a dead-letter exchange, and we alert on dead-letter depth and on the age of the oldest message."
• "Publishers set mandatory, so a missing binding shows up as an error, not as lost orders."

Remember this

• Exchanges route and never store. Unroutable messages are dropped unless you set mandatory or an alternate exchange.
• Quorum queues replicate to a majority with Raft; classic mirroring is gone in 4.0.
• Confirms on the publisher side and manual acks on the consumer side give at-least-once delivery.
• The default prefetch is unlimited. Set one.
• Dead-letter on reject, TTL, length limit or delivery limit, or those messages vanish.