A broker is a wire. Now you need to decide who gets which messages — and how. Direct, fanout, topic-pattern, work queue, dead-letter: five patterns that turn "we have Kafka" into "we have a working event-driven architecture."
M.26 set up Kafka. Brokers, partitions, consumer groups, ISRs — the infrastructure. The next question is also the harder one: how do you actually shape the flow of events through the system? Which events go to which topics? Which consumers receive which messages? What happens when a message can't be processed? Owning a broker doesn't mean having an architecture. Without explicit decisions about routing, topology, and failure handling, you end up with the worst of both worlds: the operational cost of a broker and the coupling of a monolith.
events topic is operationally simple but forces every consumer to filter. Hundreds of topics give surgical subscriptions but explode your topic count and schema-registry burden. Neither extreme is right; the trade-off is the discussion.This module is the field guide. §02 covers routing (the four mechanisms brokers use to decide who gets what). §03 covers patterns (the five canonical message-flow shapes — broadcast, work queue, request/reply, pipeline, DLQ). §04 is an interactive routing playground where you click a routing strategy and watch a message flow through the topology. §05 covers topic design — the granularity decision that quietly shapes every team's work for years.
Every broker has to answer the same question for every message: "of all the queues / topics / subscribers that exist, which ones should this message land in?" Four mechanisms have emerged as the canon — formalized by RabbitMQ's exchange types in the early 2000s, then adopted (under different names) by every broker since. Knowing them is the literacy: every routing decision in every broker is some combination of these four.
orders.us.created); binding key uses wildcards. The most expressive mechanism: lets consumers subscribe to slices of the event stream without knowing every event type. Used when event taxonomies have natural hierarchies (region, type, severity).RabbitMQ exposes all four as exchange types you configure explicitly: direct, fanout, topic, headers. Kafka takes a different conceptual path — there are no exchanges, only topics. The same routing intents are expressed via topic design: one topic per direct route, one topic with many consumer groups for fanout, multiple topics consumed by one consumer for topic patterns. The mechanisms are the same; the spelling is different.
| ROUTING INTENT | RABBITMQ-STYLE | KAFKA-STYLE |
|---|---|---|
| Direct | Direct exchange + one queue per routing_key |
One topic per event type: order-created, payment-completed. Consumers subscribe to specific topics. |
| Fanout | Fanout exchange + many bound queues | One topic + multiple consumer groups. Every group reads every message independently — analytics, billing, search, etc. |
| Topic pattern | Topic exchange + queues with wildcard bindings (orders.*.created) |
Consumer subscribes to multiple topics via regex (subscribe(Pattern.compile("orders\\..*\\.created"))) or filters in application code. |
| Work queue | One queue + multiple competing consumers | One topic + one consumer group with multiple consumers. Each partition assigned to exactly one consumer. |
The choice between RabbitMQ-style and Kafka-style isn't about which is "better." It's about where the routing logic lives. In RabbitMQ, the broker is opinionated — exchanges and bindings do the routing for you, and consumers just bind queues with the patterns they want. In Kafka, the broker is minimal — it just stores topics; consumer applications express their subscription intent. The first is easier when routing rules change frequently; the second is more flexible when you want consumers to apply complex filtering or stateful logic.
Routing is the mechanism. Patterns are the shapes you build with that mechanism — the five canonical message flows that virtually every event-driven system uses some combination of. Most distributed-systems books treat these as obvious or scatter them across chapters; we'll line them up here so the catalog is one page in your head.
A producer publishes an event; N independent consumers each react to it. The producer doesn't know how many consumers exist or what they do. Each consumer reads the whole stream at its own pace. The bedrock pattern of event-driven architectures. Implemented via fanout exchange or multiple consumer groups in Kafka.
Producer publishes tasks; a pool of workers competes to consume them. Each task is processed by exactly one worker. Load balances naturally — fast workers grab more. The default for parallel processing: image encoding, PDF rendering, ML inference jobs. Implemented via single queue + many consumers (RabbitMQ) or single consumer group with N consumers (Kafka).
Producer publishes a request with a correlation ID and a reply-to address; the responder publishes back to the reply queue, including the correlation ID. The original requester matches incoming replies to outgoing requests. Brings async benefits to RPC-style interactions — but adds complexity. Use sparingly; if you find yourself doing this often, your services might want to be more event-shaped.
A message flows through multiple processing stages, each reading from one topic and writing to the next. Each stage scales independently; backpressure naturally builds in topic depth. The workhorse of stream processing — ETL pipelines, ML feature engineering, data enrichment. Kafka Streams and Apache Flink are purpose-built for this.
Messages that fail processing repeatedly are routed to a separate DLQ instead of being retried forever (or dropped silently). An operator inspects the DLQ, fixes the data or the code, and either replays or discards. The pattern that prevents one bad message from jamming the entire pipeline. Effectively a "broken queue" that's part of every healthy system.
Most production systems use multiple patterns simultaneously. A user-signup event might be pub/sub-broadcast (everyone reacts), then trigger a work-queue task (send the welcome email — pool of workers), with a DLQ catching the inevitable malformed entries. Recognizing which pattern each part of your system is using — and noticing when a pattern is being used badly — is the skill. The next section is the playground where you watch each one play out.
Five tabs. Each is a complete topology for one of the canonical patterns. Hit Send with different routing keys; watch the message flow through the exchange, evaluate its rules, and land in the matching queues. The delivery counts on the right tick up — or don't — depending on the routing logic.
The most consequential pub/sub decision isn't routing strategy — it's topic granularity. Coarse-grained topics (one giant events stream) and fine-grained topics (one per event type) look almost identical at the start. They diverge wildly at scale. Most teams pick one extreme, then spend a year migrating to the other. Knowing the trade-off up front saves the migration.
events. Consumers filter."A single topic (or a handful) carries every kind of event. Consumers read everything and discard what they don't care about. Simple to set up, very few topics to manage, easy to add new event types without coordination. Falls apart at volume: every consumer pays the cost of every event, even ones they ignore.
Every distinct event type (order-created, payment-completed, user-signed-up) gets its own topic. Consumers subscribe only to what they actually need. Efficient at scale — no filtering overhead. Operational tax grows: hundreds of topics, schema registry per topic, partition planning per topic, more knobs to mis-tune.
The honest compromise most large systems converge on: topic-per-bounded-context, hierarchical names. One topic for orders (orders), one for payments (payments), one for users (users) — not one for every single event subtype. Each topic carries multiple event types within the same domain, distinguished by a type field in the schema. Consumers can subscribe to orders as a stream and pattern-match on event type internally. This gives you most of the efficiency benefits of fine-grained without the operational tax of topic-per-event.
Two related concerns ride on top of this. First, the schema registry: a service that holds every event schema (typically Avro, Protobuf, or JSON Schema) with versioning. Producers register schemas before publishing; consumers fetch them before deserializing. This is what keeps producers and consumers in sync as the system evolves. Confluent Schema Registry is the canonical implementation; AWS Glue Schema Registry and Apicurio are alternatives. Without a schema registry, every breaking change becomes a cross-team coordination event.
Second, event versioning and backward compatibility: when you change an event's shape, old consumers shouldn't crash. The rule of thumb is "add fields, never remove or rename" — additive changes are safe; destructive ones aren't. Mature event-driven orgs treat their event schemas like public APIs — because they are. Other teams build on top of them; breaking them ripples through the system. Every change is a release.
The terms every event-driven architecture review, every broker doc, and every retro on "why did this break" assumes you know.
orders.us.created) and matched against binding-key patterns.Test the wiring intuition. Click an answer; explanation drops in instantly.
Perfect. Routing, patterns, topic design — all yours. Now you can shape the broker into an actual architecture. Next up: events as the source of truth.
The wiring vocabulary you'll bring to every event-driven design review for the rest of your career.
Direct, fanout, topic-pattern, headers. Every broker exposes some combination of these. Picking the right one for each event type is the first design decision.
Pub/sub, work queue, request/reply, pipeline, DLQ. Production systems run several at once. The skill is recognizing each in the wild and knowing when to reach for which.
Coarse is easy until it isn't. Fine is right until the operational tax bites. Topic-per-bounded-context is usually the right middle ground. Add a schema registry from day one.