Spaghetti in a box: the ESB pattern that follows you to Kafka

Swapping an enterprise service bus for Kafka does not kill the antipattern that made the bus painful. The antipattern was never the broker; it was putting routing, transformation and business logic in the pipe instead of in the services at each end. Move that logic onto Kafka unchanged and you get the same tangle, now spread across topics and harder to see. The fix is ownership, not technology: smart endpoints, dumb pipes.

What was actually wrong with the enterprise service bus?

Not the messaging. Most buses moved bytes reliably enough. What rotted was everything the integration team bolted into the middle: content-based routing, format translation, orchestration, enrichment and business rules, all living in a shared tier that every project had to queue up and negotiate with. Jim Webber's name for the result, quoted in the Fowler and Lewis microservices essay, was that ESB stands for Erroneous Spaghetti Box. The corrective in that same 2014 piece sits among its nine characteristics of a microservice architecture:

Smart endpoints and dumb pipes.

Put the intelligence in the services that produce and consume messages, and keep the transport dumb — something that stores and forwards and guarantees order, and knows nothing about what an order is. The damage from doing the opposite was operational, not aesthetic. Because the smarts lived centrally, any change one team needed became a change to shared infrastructure everyone else depended on, and release cadence fell to the speed of the slowest reviewer on the bus team. The failure blast radius was the whole estate. That last property is the one that follows you.

Why does the same shape reappear on Kafka?

Because the broker was never the thing you were buying, and Kafka is a very good broker. The temptation is to make it clever: a central topic namespace, a shared stream-processing layer, and one integration group that owns the transforms between every producer and every consumer. That is the enterprise service bus again, wearing a partition key. ThoughtWorks has been explicit about the trap, placing Recreating ESB antipatterns with Kafka on the Hold ring — one of the four rings — of its Technology Radar, warning that concentrating transformation logic ends up:

creating a significant dependency on a centralized team.

Nobody sets out to rebuild a bus. It starts with one reasonable stream query to reshape an order event for a downstream that cannot be changed today. Then a second. Two years later there are hundreds of transforms in the middle, a team that owns them, and a backlog every other team is stuck behind. We are not innocent here: on an early build we let a shared enrichment service creep into the pipe because it was faster than coordinating two teams, and unpicking it cost us the better part of a quarter.

The confusing part is that some centralization is fine. A schema registry, topic naming conventions, the security model, cost governance — those are dumb-pipe concerns and they belong to a platform team. The line is drawn at meaning. The moment the central layer starts knowing what an order is, or how a returnable maps to a deposit, it has crossed from plumbing into business logic, and every consumer is now downstream of a team that does not own their domain.

What does the failure look like at 2 a.m.?

The same way it did on the bus: everywhere at once. An analysis of ESB failure modes names three of them in a single sentence, any one of which is enough:

A misconfigured route, a memory leak, or a bug in a transformation rule could cascade to every system connected to the bus.

Move those transforms into a shared Kafka layer and the property is preserved exactly. In a beverage route-to-market estate, order capture, stock, loyalty, returnables, delivery and analytics all ride the same event stream. Put a currency-rounding bug or a careless schema change in the central transform and you have not broken one consumer — you have shipped wrong numbers to all of them, quietly, until a depot phones in. It is the same treacherous mode we described in batch jobs that fail for 500 seconds without leaving a log, except the blast radius is every system on the stream instead of a single job.

So who actually owns the schema?

This is the real diagnosis, and it is an org-chart question dressed as a technical one. In the antipattern, one central team owns the meaning of the data — its shape, its transforms, its projections — while the domain teams who actually understand orders or returnables own none of it. That is why the bottleneck forms and why the blast radius is total: everyone waits on the middle, and the middle can break everyone. It is the same dependency that bites when a vendor owns your loyalty rules and a rollout slips from 90 days to 12 months, or when a commerce platform reaches end-of-life and the teams who must migrate cannot, because the thing they need to change belongs to someone else.

The direction out is not another broker; it is moving ownership to the edges. Producers own and version their schemas and publish a stable contract; each consumer owns its own projection and reshapes raw events in its own service, on its own release schedule; the pipe just stores and forwards. That single inversion — the default behind how we lay out our event architecture — is what stops a shared stream curdling back into a bus.

None of this is a Kafka setting you can toggle. The antipattern is organizational, so it re-forms wherever a well-meaning team offers to just handle the mapping for everyone. The estates that stay fast over the next few years will be the ones that treat the location of transformation logic as a named architectural decision with a named owner, and defend the dumb pipe like the boundary it is.