Why Do You Need a Message Broker?

01 — The scenario

Real situation

You have a booking management SaaS. When a user confirms a booking, your system needs to do three things:

Send a confirmation email, generate the invoice and log the operation in the audit trail.

Nothing unusual so far. The problem is how you do it.

02 — Without a broker: everything synchronous

The most straightforward solution is having the same server that receives the request do all the work sequentially:

1. User clicks “Confirm booking” → POST /booking
2. The server saves the booking to the database
3. Calls the email service and waits for a response — 1.2s
4. Generates the invoice as PDF — 0.8s
5. Writes the audit log — 0.3s
6. Only now responds to the user — ~2.5s total

The problem isn’t that it takes 2.5 seconds.

The problem is that while processing that booking, the server is blocked. If 50 users do the same thing at the same time, you have 50 threads busy generating PDFs and sending emails. Login starts lagging. The dashboard won’t load. Everything degrades.

This isn’t a Kafka problem, or a RabbitMQ problem, or any specific technology problem. It’s a design problem: you’re using the same resource (your server) for two things with different rhythms — responding to the user (must be fast) and processing heavy tasks (can wait).

03 — With a broker: decoupling

The idea is simple: separate “receiving the request” from “processing it”. The server receives the booking, saves it, enqueues a message saying “there’s pending work” and responds instantly. Another process handles the heavy lifting separately.

1. User clicks “Confirm booking”
2. The server saves the booking to the database
3. Sends the broker a message: "booking.created · ID 4821"
4. The broker stores the message in a queue
5. The server responds to the user → ~120ms
6. The broker delivers the message to another process that sends the email, generates the invoice and writes the log

Flow: Users → Producer → Broker → Consumer

04 — What do you gain?

Fast response. The user’s request responds in milliseconds, not seconds. The experience improves immediately.

Free server. The process handling HTTP doesn’t waste resources on heavy tasks. Login, dashboard, API — everything keeps responding normally.

Independent scaling. If you need to process more emails, you scale the consumers without touching the main server. If a consumer crashes, messages wait in the queue.

Resilience. If the email service fails at 3AM, the task isn’t lost. The broker keeps it. When the service comes back, it processes everything pending.

05 — The rules of the game

A broker doesn’t simplify your system. It makes it more robust as it starts to grow. But it introduces complexity you need to manage:

The flow is asynchronous. The user receives an “OK” before the email has been sent. You need to design for that: intermediate states, progress notifications, visual feedback.

Idempotency. If the consumer processes a message and crashes before confirming it, the broker will retry. Your logic must handle receiving the same message twice without duplicating the invoice or sending two emails.

Retries and DLQ. What happens if a message fails once, twice, five times? You need a strategy: retries with backoff, and a Dead Letter Queue where hopeless messages land — for a human to review.

Observability. Now you have a queue in the middle. You need to monitor lag (how many messages are pending), error rates and your consumers’ throughput.

A broker is not a silver bullet. It’s a design tool. It lets you separate the urgent from the important, and scale each part independently.

06 — Which broker should I use?

This article is intentionally agnostic. The decoupling pattern with a broker applies equally with RabbitMQ, Kafka, Amazon SQS or any other. The question of which one to choose depends on a second level of decisions: do you need persistence? event replay? multiple consumers reading the same message? guaranteed ordering?

That’s exactly what we explore in part two, where we compare a traditional message queue with a distributed log like Kafka — and why for certain scenarios the difference is critical.