CODEWITHSUNDEEP
event-sourcing
Hector Ordonez
Hector Ordonez

Reputation: 1104

How should projectionists run in an event sourced system?

I have been reading on event sourcing for a while. The information regarding how to actually build projections seems... scattered.

From what I gathered, we have a projectionist, which manages projectors, which build and update projections.

How should a system run a projectionist? Should it be one process, which handles all projectors? Or should it run one projectionist per projection, independent from each other?

Upvotes: 1

Views: 67

Answers (2)

iTollu
iTollu

Reputation: 1069

I am currently building an event-sourced system.

It uses PostgreSQL as EventStore and AWS Lambda as runtime.

Commands are processed by HTTP API, then events are stored to EventStore. I have concept of a stream type. It more or less matches an entity. Like, stream type "User", stream type "Account", stream type "Product".

For each stream type EventStore sends an SNS notification.

Projections are implemented as AWS Lambda functions triggered by SQS messages. Every projection has a dedicated SQS queue. This queue is subscribed to the stream types it needs.

A projection may need multiple stream types, one of them being initiating. I track all stream ids in a projection and latest processed stream versions.

In this setup it really doesn't matter if a projection is a separate lambda, or all projections are handled by the same lambda. They are really easy to deploy.

If in your case you run projectors in, say, a Docker container - I guess it will be easier for you to have as little deployment units as possible. So, in this case a dockerized application will be subscribed to multiple queues and will route a events internally to dedicated projectors.

I'll be happy to share more technical details, if it helps.

Upvotes: 0

Tore Nestenius
Tore Nestenius

Reputation: 19941

It depends.

A central projectionist handling all projections can work well initially. But if some views take longer to update than others, it can become a bottleneck.

Running a separate projectionist per view allows each model to process events independently. If one is slow or fails, the others can continue without disruption. This also makes it easier to introduce new models that can catch up in the background and lets you rebuild individual read models as needed.

Upvotes: 1

Related Questions