In Event Sourced systems, how do you deal with adding past events. Is ES right choice for such scenario?

Question

All of the cases of Event Sourced systems I encountered (both real-world cases and example scenarios considered in blogs) were assuming the events are stored in the system the very moment they happen in real world. What if we consider a technical or human-caused delay? Consider an employee who's trying to keep up with the notes left on his desk. He needs to add "past events". Like, "this happened two days ago, but nobody marked it in the system yet"

Let's consider a scenario of "small rental car company" application. The events would be:

- CarRented (rentalStartDate, ...)
- CarReturnedByCustomer (returnDate, odometerReading, milesDriven ...)
- CarServicedByMechanic (serviceDate, cost, ...)

The application is used by the only employee of the company. When renting and returning the car, they update the application immediately because the app generates the documents printed to the customer etc. However when the car returns from maintenance they're too busy to mark it immediately, and want to do it the next day. Or maybe there is an integration with the external app that sends the batch of daily data overnight.

Basically we end up with the events sequence not being aligned with the actual order of when the events happened:

- CarRented (date: 2022-08-20)
- CarServicedByMechanic (date: 2022-08-16) <-- note the date here!
- CarReturnedByCustomer (date: 2022-08-22)

Now consider the projection which tracks the "mileage since last service" for each car:

On CarReturnedByCustomer: model.MileageSinceLastService += event.MilesDriven
On CarServicedByMechanic: model.MileageSinceLastService = 0;

this projection won't work properly in our scenario, where events are out of order with the real world.

How do you deal with such cases? Is that a heuristic against using ES for such scenarios? However when I think of it, multiple complex systems might at some point face the need to deal with imported data, thus - adding events "post-fact"

Answer 1

In Event Sourced systems, how do you deal with adding past events?

Exactly the same way you deal with past events in other designs: you model time.

---

Part of the confusion here is that "event" is used to represent a number of different ideas, and "event sourcing" adds even more idea to that list.

(A lot of this happens in a context where "everyone" is familiar with the fact that small shifts in context can produce large changes in the semantics of a label; but then those same labels escaped to the community at large, and we all learn that re-using the label wasn't actually such a great idea.)

We need to not confuse events ("the domain specific representations of patch documents we store in our durable memory") with events ("messages carrying copies of information from outside our domain model").

---

Essentially, what you are discovering here is two ideas: one is that you can't rely on messages to arrive in the "correct" order.

The other is that message metadata (timeWritten, timeSent, timeReceived, timeProcessed) are not necessary good proxy measures for domain data (timeServiced).

So the "right" answer is to ensure that your domain messages carry all of the information that they need to, and to ensure that your domain logic includes the branches needed to correctly integrate this new information with what has already been written down.

But those are the hard parts; the event sourcing work is the same problem that you had before "ok, how do we write these changes down so that we can remember them later?". Are you going to use a new variant of patch document (aka a new "event") or re-use what you already have in production?

---

Bad news: using event sourcing or not, your design needs to understand the nature of the data entry.

For example, if your system is designed for a context where people take a bunch of notes, and then enter them as a batch, then you need to be thinking through the implications of that when designing the system.

The machine fits the man.