EVENT SOURCING — THE PRIVILEGED MODEL FOR INDIRECTION — PART 3

  • You can read the part 1 here.
  • You can read the part 2 here.

In the last parts it was presented an evolutionary path towards an event driven architecture to support multiple read models. We ended with an architecture with a relational database that supports write operations where the business invariants are ensured, an event queue, a broker to dispatch messages, and an Integrator to update changes to read-only databases (Image 1). In this part 3 It will be presented an Event Sourcing alternative and be discussed some nuances in the adoption of such solution.

Image 1 — Final Architecture from part 2.

Event Sourcing is a radically different way to persistence. An object state is persisted by storing the sequences of state changing events.

The realization of an Event Sourcing model is very simple, just use the state change events as the persistence model.

Abandoning the relational representation of the state allows to simplify the write model because we only need to maintain the event model instead of two different models. Using events as the state of the application has several implications in the code base. The first obvious change to note is the way the in-memory entities are hydrated. No more object-relational mapping is needed. An entity with the current state is no more than a projection of the event stream that can be created by a projector function: f(events) -> state. A naive implementation of a repository that hydrates an event sourced entity looks very much like this:

class HotelRepository
{
private readonly EventStreams _streams;

public Hotel Find(Guid id)
{
var events = await _streams.GetAllEventsFromStream(streamId:id);
var hotelState = new HotelState();

// projector
@events.LeftFold(hotelState, (state, evt) => state.Apply(evt));
return new Hotel(id, hotelState);
}
}

The HotelState class will handle the state change events, mutating the state to match the actual state representation:

class HotelState
{
private string _address;
private List<Room> _rooms;

public HotelState Apply(HotelCreated ev)
{
_address = ev.Address;
_rooms = rooms;
return this;
}

public HotelState Apply(RoomBooked ev)
{
var room = _rooms.Find(ev.RoomId);
room.Bookings.Add(ev.Booking);
return this;
}

public HotelState Apply(BookingCanceled ev)
{
var room = _rooms.Find(ev.RoomId);
var booking = room.Bookings.Find(ev.BookingId);
room.Bookings.Remove(booking);
return this;
}

// (...)
}

Using Event Sourcing allows a better decoupling between the entity model and the persistence model. It allows to evolve the entity model without any changes in the persistence model. The relational-entity mapping (huge) problem is gone. Adding new features and new events can be done by creating new events and extending or creating a new projection used as write model, no need to create data migrations or redesign the data model. Off course this is only true without breaking changes. If we want to change the Booking app to allow to manage a wine store, that is not possible, but it doesn’t make sense anyway. Sometimes event evolution may be needed. A lot can be said about event versioning, that subject will not be addressed here, but it is one of those matters that should be treated with care. Extending events has a lot of nuances that should be very well understood to be done with success. I recommend the book draft (still in progress) that Greg Young is being writing about the subject esversioning.

To implement the business logic we can use the aggregate boundary to enforce the invariants of the application. In this case we will use the Hotel entity as the root entity of the aggregate. Any business operations like Hotel creation, bookings, cancellation, etc, are done through the Hotel entity. An application service would have the following structure:

  1. Load the Hotel entity to memory
  2. Perform in-memory changes
  3. Save changes in the database

The step 1 was already illustrated in the repository code snippet. The Hotel entity code that manages the step 2 can be something like this:

class Hotel
{
public readonly Guid Id;
private readonly HotelState _state;
private readonly IList<Event> _changes;
public IEnumerable Changes => _changes.ToArray();

public Hotel(Guid id, HotelState state){...}

public Result CreateHotel(CreateHotelArgs args)
{
ValidateArgs(args);
if(hotelAlreadyCreated())
{
// idempotence, ignore or return error, whatever suits better your
business requirements
return Result.Idempotence;
}

var hotelCreated = HotelCreated.Map(args);
Apply(hotelCreated);
return Result.Ok;
}

public Result BookRoom(RoomBookingArgs args)
{
ValidateArgs(args);
var room = _state.Rooms.Find(args.RoomId);
if(room.IsAlreadyBooked(args.StartDate, args.EndDate))
{
return Result.Error<RoomNotAvailable>();
}

var roomBooked = RoomBooked.Map(args);
Apply(roomBooked);
return Result.Ok;
}

(...)

private void Apply(Event evt)
{
_changes.Add(evt);
_state.Apply(evt);
}
}

All code inside the Hotel class runs all needed validations like ensuring the correct parameters are passed, and that no invalid state should be reached, like overbooking. But this code is not enough to ensure that concurrent operations for the same Hotel does not happen, like two customers trying to book the same room for the same week. To solve this we need a synchronization mechanism somewhere. We can solve it in the step 3, when changes in the database are saved. We can use an optimistic concurrency strategy, taking advantage of the database conditional-write capabilities. In a SQL database we can use a transaction to atomically append the events to the Events table AND update the Entity version in the Entity table IF and only IF the current version of the Entity is the expected. In a race condition the expected version changes and the second operation fails with a concurrency error. The code bellow is a (very) simplified example of an optimist concurrency implementation. You can check a full implemented version of a SQL event store here.

class EventStoreDb
{
public void SaveChanges(
Guid streamId,
int expectedVersion,
Event[] changes)
{
int newVersion = expectedVersion + changes.Length;
try
{
using (var transaction = new TransactionScope())
{
using (var con = new SqlConnection(_connectionString))
{
await con.ExecuteAsync(@"insert into [Events] values (
@streamId,
@entityId,
@eventSequence,
@eventType,
@eventPayload)";)

await con.ExecuteAsync(@"update [Entity]
set [EntityVersion] = @newVersion
where [EntityId] = @streamId
and [EntityVersion] = @expectedVersion");

transaction.Complete();
}
}
}
catch (SqlException e)
{
if (IsConcurrencyError(e.Number))
{
throw new ConcurrencyException(streamId, expectedVersion);
}
else
{
throw;
}
}
}
}

With only one extra field in the Event table we can change dramatically the architecture. Let’s add a GlobalVersion field, an incremental int to the Event table. This field serves as a cursor of the global state of the App. We’ll see how we can take advantage of these changes in the overall architecture.

We removed the Integrator as a central piece and moved the responsibility of the event handling for each read-model. There is also no need to support any special bootstrap mode. All consumers when get plugged into the system, only need to have their own Global Version of the state to 0 and request all Events from position 0. A request to new events should be made to the event store, by each consumer, with the adequate time periodicity. It also has the benefit of giving event batching out-of-the-box as any request always returns all the events from the last position.

In the publication side, there is no need to support a broker, neither a two-phase commit, the event store is the single point of communication between producer and consumers.

Another benefit is that it is very easy to know what is the state for each read-model as it is provided a global version. Intermittent networking failures recovery is seamless, any read model can get disconnected from the event store, but when it gets connected again, it only needs to request all the events since the last known state.

The event log provides a single source for data synchronization, in a time-series ordered state without any loss of data. This is a very powerful concept.

Final Considerations

In this series of 3 articles it was illustrated an architectural path to support multiple models for the same application. The necessity to have several models is increasing with the demand for richer apps with complex use-cases. Event Sourcing is a very powerful model that proves to be a natural fit in data distribution scenarios. It not only provides powerful features like an audit-proof log, but allows a simpler architecture in certain distributed scenarios.

I cannot finish without stating a very important note about a subject I find to be one of the most common (and dangerous) misconception about Event Sourcing:

Event Sourcing is not suitable to be used as a high-level architectural pattern.

Borrowing Eric Evans terminology, Event Sourcing should be used Inside a bounded context. In the example of the booking application, all the diagrams we presented are part of the same Booking bounded context. If we zoom out to a higher level architecture diagram of the system we would find something like this:

The Event Sourcing events are internal to the Booking Service. They are part of the implementation details of that bounded context. Other events can be used to communicate between different modules but those are other types of events, integration events, not the same events used with Event Sourcing. There are lots of integration use-cases that take advantage of event driven approaches with success. But those patterns are not Event Sourcing. The Kafka based solutions that are in vogue nowadays and claim being using Event Sourcing, are using different architectural patterns. In fact Kafka is not suitable to be used as an event store because it lacks two fundamental features: atomic conditional writes and event indexing. Without those two features it is not possible to implement a write model based on a stream of events.

End.

Software developer @Sky. Developing software on the shoulders of giants.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store