Choosing a Message Broker: RabbitMQ vs. SQS vs. Kafka

"Just put a queue between them" is one of the most common pieces of architecture advice, and also one of the most underspecified. A queue could mean RabbitMQ, Amazon SQS, or Kafka — three systems that all technically let one service hand work to another asynchronously, and that otherwise share very little. They differ in delivery model, ordering guarantees, operational cost, and whether "replaying" a message even makes sense as a concept. Picking between them isn't a matter of taste; it's a matter of which failure modes you're willing to design around.

RabbitMQ: the smart broker

RabbitMQ implements the classic "smart broker, dumb consumer" model. The broker itself understands routing: producers publish messages to an exchange, and the exchange decides which queue (or queues) each message lands in based on a binding. A direct exchange routes by exact key, a topic exchange routes by wildcard pattern ("orders.eu.*"), and a fanout exchange broadcasts to every bound queue regardless of key. This makes RabbitMQ excellent at expressing routing logic declaratively — "send this to the fraud-checks queue and the audit-log queue, but only if it's a EU transaction" — without the consumers needing to know anything about who else is listening.

Consumers acknowledge messages explicitly, and an unacknowledged message gets redelivered, which gives you solid at-least-once delivery. RabbitMQ also supports priority queues, per-message TTLs, and dead-letter exchanges out of the box — all useful when the routing logic itself is part of your business problem. What it isn't built for is throughput at Kafka's scale, or treating message history as a durable, replayable asset — once a message is acked and removed from the queue, it's gone.

Amazon SQS: the boring queue that just works

SQS deliberately narrows the problem down to almost nothing: it's a durable queue, fully managed, with no brokers to patch, no cluster to size, and no partitions to rebalance. You send a message, it sits there until a consumer receives and deletes it. There's no routing layer — if you need fan-out, you put an SNS topic or EventBridge bus in front of it — and there's no concept of replay. Once a message is deleted, it's gone for good.

Instead of explicit acks, SQS uses a visibility timeout: when a consumer receives a message, it becomes invisible to other consumers for a configurable window. If the consumer finishes processing and calls delete before that window expires, the message is removed for good. If it crashes, or simply doesn't call delete in time, the message reappears and another consumer picks it up. It's a subtle but important shift in mental model from RabbitMQ's ack/nack — you're racing a clock, not sending a signal. Standard SQS queues give you at-least-once delivery and near-unlimited throughput but no ordering guarantee; FIFO queues add strict ordering and exactly-once processing, scoped per message group, at a meaningful throughput cost.

The trade-off is honest: you give up routing sophistication and message replay, and in exchange you give up almost all operational overhead. For a huge fraction of "decouple these two services" problems on AWS, that trade is exactly right.

Kafka: a log, not a queue

Kafka looks like a message broker from the outside but is built around a fundamentally different primitive: a durable, ordered, append-only log, partitioned across brokers for throughput and replicated for durability. Consumers don't delete what they read — they track an offset, a position in the log — which means the same message can be read by ten different consumer groups, and any one of them can rewind and re-read history whenever it wants.

# A "queue" in Kafka is really a durable, replayable log
producer.send("orders.created", key=order.customer_id, value=order)

# Consumers track their own offset — replay is just rewinding it
consumer.subscribe(["orders.created"])
consumer.seek_to_timestamp(yesterday_9am)  # reprocess history
for record in consumer.poll():
    handle(record.value)
    # offset commits independently of RabbitMQ/SQS-style acking

That single property — replay — is what separates Kafka from RabbitMQ and SQS conceptually, not just operationally. It turns "reprocess last week's events with the fixed bug" or "backfill a brand-new downstream service from historical data" from a painful special case into a normal operation. Combined with strict ordering within a partition and an ecosystem built for stream processing — Kafka Streams, ksqlDB, Kafka Connect — Kafka is less a work queue and more a backbone for events that multiple systems need to observe, replay, and derive state from. The cost is real, though: running Kafka well means managing brokers, partitions, replication factors, and consumer group rebalancing, or paying for a managed offering that handles it for you.

The axes that actually decide it

Stripped of marketing, the choice comes down to four questions asked in order.

• Throughput. RabbitMQ comfortably handles tens of thousands of messages per second per node; SQS scales near-linearly with almost no ceiling you'll hit in practice; Kafka is built for the same range as SQS and beyond, but earns that scale through partitioning you have to design for up front.
• Ordering, and at what granularity. If you need global strict ordering, none of these three give it to you for free at scale. What you can get is ordering within a scope: a RabbitMQ single-consumer queue, an SQS FIFO message-group, or a Kafka partition. The real question is whether your ordering requirement is naturally per-entity (per order, per user) — if so, all three can express it; if it's truly global, you'll need to serialize processing somewhere regardless of broker.
• Operational overhead and team expertise. SQS costs you almost nothing to run. RabbitMQ needs a team comfortable operating a stateful broker (or budget for a managed one). Kafka demands the most — partition planning, retention policy, consumer group behavior — and punishes teams that adopt it without that investment.
• Do you need to replay history? This is the sharpest dividing line. If "replay last month's events" is a real requirement — not a nice-to-have — Kafka (or a Kafka-compatible log like Redpanda or Amazon MSK) is close to the only reasonable answer. RabbitMQ and SQS treat a consumed message as gone; retrofitting replay onto either means bolting on your own event store, at which point you're halfway to building Kafka yourself.

So which one

Default to SQS if you're already on AWS and the problem is genuinely "decouple A from B" with no complex routing and no need to replay history — it's the choice that lets your team spend its time on the product instead of the broker. Reach for RabbitMQ when the routing logic itself is nontrivial — multiple consumers need different slices of the same stream based on content, priority, or pattern — but you don't want the operational commitment of a distributed log. Reach for Kafka once you need durable history, not just a work queue: multiple independent consumers reading the same events at their own pace, replay for backfills and bug fixes, or throughput and stream-processing needs that outgrow a traditional broker. The mistake to avoid in either direction is choosing Kafka by default for its resume value, or choosing SQS and then spending the next year building replay and fan-out on top of it by hand.