SproutStackSproutStack home
IntermediateRabbitMQ

Upgrading a cluster that uses mirrored queues

Payments run on classic mirrored queues. The cluster is moving to RabbitMQ 4.

The setup today

A three-node RabbitMQ 3.13 cluster carries payment commands. To survive a node failure, the payment queues are classic queues mirrored to every node by policy.

Current configuration

rabbitmq 3.13   nodes: 3
policy ha-all: ha-mode=all  ->  ^payments\.
queue: payments.capture   type: classic (mirrored)

What you know

  • Consumers use manual acks; publishers use confirms.
  • A bad message occasionally makes a consumer crash.

The new requirement

The platform must upgrade to RabbitMQ 4.x for security support. Payments must still survive the loss of any one node after the upgrade.

Make the call

2 questions · first-try answers earn the most XP

1. What do you do with the payment queues?
2. After the move, the occasional crashing message behaves differently. Why, and what must you add?
What a strong answer also covers
  • Quorum queues need a majority online: with three members, one node can be down; plan maintenance one node at a time.
  • Check the upgrade path: move off mirrored queues before upgrading, since 4.x cannot run them.
  • Expect lower throughput per queue than an unreplicated classic queue; load-test the payment path.
  • Since 4.3, a basic.nack with requeue does not count toward the delivery limit, so poison handling should reject without requeue.

The background

More drills