Event Sourcing: why using a message broker is a bad idea
I’ve seen many implementations of Event Sourcing that use a message broker to publish events to the read models. I’ve used that pattern and contributed to ES “frameworks” that implement that pattern. I think It is a bad idea. I’ll explain why:
1) Writing to the event store AND publishing events to a broker needs to be atomic. In most cases a distributed transaction using two phase commits is not an option. We need to create some mechanism to deal with faulty connections between publisher and broker:
To reliably/atomically update the event store and publish events to the broker, we can use the Transactional Outbox Pattern:
The outbox pattern uses a message relay process to publishes the events inserted into event store to a message broker. It can be done in two flavours:
a) Use a separate service to handle the relay process allowing a simpler publisher, but requiring a new moving piece.
b) Implement the relay process logic as part of the publisher code. Less moving pieces but adds complexity to the publisher.
Example of a publisher that retries to publish undelivered events to the broker whenever a new event load request comes
