Design of notification events

Question

I am designing some events that will be raised when actions are performed or data changes in a system. These events will likely be consumed by many different services and will be serialized as XML, although more broadly my question also applies to the design of more modern funky things like Webhooks.

I'm specifically thinking about how to describe changes with an event and am having difficulty choosing between different implementations. Let me illustrate my quandry.

Imagine a customer is created, and a simple event is raised.

<CustomerCreated>
  <CustomerId>1234</CustomerId>
  <FullName>Bob</FullName>
  <AccountLevel>Silver</AccountLevel>
</CustomerCreated>

Now let's say Bob spends lots of money and becomes a gold customer, or indeed any other property changes (e.g.: he now prefers to be known as Robert). I could raise an event like this.

<CustomerModified>
  <CustomerId>1234</CustomerId>
  <FullName>Bob</FullName>
  <AccountLevel>Gold</AccountLevel>
</CustomerModified>

This is nice because the schema of the Created and Modified events are the same and any subscriber receives the complete current state of the entity. However it is difficult for any receiver to determine which properties have changed without tracking state themselves.

I then thought about an event like this.

<CustomerModified>
  <CustomerId>1234</CustomerId>
  <AccountLevel>Gold</AccountLevel>
</CustomerModified>

This is more compact and only contains the properties that have changed, but comes with the downside that the receiver must apply the changes and reassemble the current state of the entity if they need it. Also, the schemas of the Created and Modified events must be different now; CustomerId is required but all other properties are optional.

Then I came up with this.

<CustomerModified>
  <CustomerId>1234</CustomerId>
  <Before>
    <FullName>Bob</FullName>
    <AccountLevel>Silver</AccountLevel>
  </Before>
  <After>
    <FullName>Bob</FullName>
    <AccountLevel>Gold</AccountLevel>
  </After>
</CustomerModified>

This covers all bases as it contains the full current state, plus a receiver can figure out what has changed. The Before and After elements have the exact same schema type as the Created event. However, it is incredibly verbose.

I've struggled to find any good examples of events; are there any other patterns I should consider?

Answer 1

You tagged the question as "Event Sourcing", but your question seems to be more about Event-Driven SOA.

I agree with @Matt's answer--"CustomerModified" is not granular enough to capture intent if there are multiple business reasons why a Customer would change.

However, I would back up even further and ask you to consider why you are storing Customer information in a local service, when it seems that you (presumably) already have a source of truth for customer. The starting point for consuming Customer information should be getting it from the source when it's needed. Storing a copy of information that can be queried reliably from the source may very well be an unnecessary optimization (and complication).

Even if you do need to store Customer data locally (and there are certainly valid reasons for need to do so), consider passing only the data necessary to construct a query of the source of truth (the service emitting the event):

<SomeInterestingCustomerStateChange>
  <CustomerId>1234</CustomerId>
</SomeInterestingCustomerStateChange>

So these event types can be as granular as necessary, e.g. "CustomerAddressChanged" or simply "CustomerChanged", and it is up to the consumer to query for the information it needs based on the event type.

There is not a "one-size-fits-all" solution--sometimes it does make more sense to pass the relevant data with the event. Again, I agree with @Matt's answer if this is the direction you need to move in.

Edit Based on Comment

I would agree that using an ESB to query is generally not a good idea. Some people use an ESB this way, but IMHO it's a bad practice.

Your original question and your comments to this answer and to Matt's talk about only including fields that have changed. This would definitely be problematic in many languages, where you would have to somehow distinguish between a property being empty/null and a property not being included in the event. If the event is getting serialized/de-serialized from/to a static type, it will be painful (if not impossible) to know the difference between "First Name is being set to NULL" and "First Name is missing because it didn't change".

Based on your comment that this is about synchronization of systems, my recommendation would be to send the full set of data on each change (assuming signal+query is not an option). That leaves the interpretation of the data up to each consuming system, and limits the responsibility of the publisher to emitting a more generic event, i.e. "Customer 1234 has been modified to X state". This event seems more broadly useful than the other options, and if other systems receive this event, they can interpret it as they see fit. They can dump/rewrite their own data for Customer 1234, or they can compare it to what they have and update only what changed. Sending only what changed seems more specific to a single consumer or a specific type of consumer.

All that said, I don't think any of your proposed solutions are "right" or "wrong". You know best what will work for your unique situation.

Answer 2

Events should be used to describe intent as well as details, for example, you could have a CustomerRegistered event with all the details for the customer that was registered. Then later in the stream a CustomerMadeGoldAccount event that only really needs to capture the customer Id of the customer who's account was changed to gold.

It's up to the consumers of the events to build up the current state of the system that they are interested in.

This allows only the most pertinent information to be stored in each event, imagine having hundreds of properties for a customer, if every command that changed a single property had to raise an event with all the properties before and after, this gets unwieldy pretty quickly. It's also difficult to determine why the change occurred if you just publish a generic CustomerModified event, which is often a question that is asked about the current state of an entity.

Only capturing data relevant to the event means that the command that issues the event only needs to have enough data about the entity to validate the command can be executed, it doesn't need to even read the whole customer entity.

Subscribers of the events also only need to build up a state for things that they are interested in, e.g. perhaps an 'account level' widget is listening to these events, all it needs to keep around is the customer ids and account levels so that it can display what account level the customer is at.

Answer 3

Instead of trying to convey everything through payload xmls' fields, you can distinguish between different operations based on - 1. Different endpoint URLs depending on the operation(this is preferred) 2. Have an opcode(operation code) as an element in the xml file which tells which operation is to used to handle the incoming request.(more nearer to your examples)

There are a few enterprise patterns applicable to your business case - messaging and its variants, and if your system is extensible then Enterprise Service Bus should be used. An ESB allows reliable handling of events and processing.