Migrating relational data to event store

Question

I am migrating an old project towards CQRS using my own implementation of the CQRS pattern. The main reason I started this migration was to get rid of the mess the n-tier architecture had caused. The project however, as many others, is using a relational data store.

The current state in this process is that I have Commands that make changes and Query objects that isolate the querying. This means that from the client to server, I have a CQRS-ish way of changing and querying data although I do not publish any events from the changes, nor do I have a denormalized read store. I should mention that I also have pretty anaemic DTOs as my "domain model". All behaviour was distributed throughout the n-tier layers with its handlers, managers and all those horrible layer things.

I want to know how I can take the next step. What I want now is to start building a domain model that is in charge of its behaviour and the reason for that is that I want to start using an event store as the source of truth which brings me to my question:

How can I migrate the data from the relation data store into the event store?

I have an extremely normalized data model with very important data, which has to be migrated. I do understand that I cannot expect to capture any intent from that data as it is dead, but what should I do with it? Should I create loads of migration commands? It would be nice to hear your experience from this.

Answer 1

Id approach this differently to the solutions offered. Ive done this migration twice on 2 different projects.

Your first step of having commands do the writes and queries do the reads is a good first step.

The next thing I would do is raise events from the same commands that do the writes. Get comfortable with the event raising.

This simply means in the command handler just build the events and push them into the EventStore.

The earlier in the projects life you start committing events, the more history you have in your app.

Now, continue to perform the writes to the normalized database in your command handlers.

Once you have a couple of command handlers raising events, the next step is quite easy. Build an event handler/denormalizer that listens for the raised event and simply performs EXACTLY the same write operation the command handler did.

All you have done now is move the WRITE responsibility into the denormalizer and away from the command handler.

Thats basically the process I would recommend, as I said, Ive done this twice and it works.

The matter of migrating EXISTING data in your data base to events is a little tricky. You have to think of that data as the current state, so all you can do I build and commit (using a some utility you write) Create events for all the data in your systems.

Eg, to migrate an existing Accounts table, you'd write an AccountCreatedEvent and commit it with all the data in each account in the table.

Answer 2

I’ve not had the opportunity to try this but something I wanted to try would work in a number of stages:

1. First map out all the events that the system would need.
2. Next introduce Views into the relational database to represent the events.
3. Write a custom utility to select from the views to build the correct event objects.
4. Have the custom utility write the event objects to the event store in correct sequence.
5. Finally make sure to refresh denormalised view models from the event store.

I would imagine that the sequence in which the events are written to the event store is a bit tricky and would require a bit of thought. But seeing as this is a migration process then as long as the final state of the domain is correct I don’t think it matters to much. If your relational tables have timestamps then this could help.

I decided that building event objects was more appropriate rather that creating commands because the relational data represents the domain for actions that have happened in the past.