CODEWITHSUNDEEP
architecturemicroservicessystem-designevent-drivenevent-driven-design
Sean Hwang
Sean Hwang

Reputation: 451

Should each microservice contain event consumer itself or separate from it in event-driven MSA?

Which one is better between
(1) having event consumer(e.g. KafkaListener) in server application of a microservice
(2) having another application only for consuming the events and calling API of the microservice which contains business logic(e.g. data CRUD)

A diagram of (1) is,
enter image description here
A diagram of (2) is,
enter image description here

At first, I thought that (1) is better because we should make it atomic for event consuming, proccessing(e.g. aggregate update), and event producing when we should implement transactions in distributed system(Saga pattern), so we should wrap all of these with local DB transaction using transactional outbox pattern for data consistency. If you don't do like this, the result state of data could be inconsistent under partial failure because intermediate context is lost when using synchronized API call, not transactional messaging. You can use messaging instead of API call after consuming the event, but then it seems better to put event consumer in the microservice at first without having separate event consumer application.

(2) looks better in that we can scale event consuming resource separately from business logic. We don't need to scale event consuming logic when there is not so many events compared to requests from another server components. Same for the opposite, we don't need to scale business logic when there is not so many real-time API calls from other microservices(or API Gateway/BFF), but many events to consume. And I also feels like a bug or failure in event consuming logic should not affect to business logic. A microservice should still handle API calls from another server components even in a failure on event consumer logic. But, it also has a disadvantage for having to manage additional server applications for every microservice.

I think each of them have pros/cons, but I'm curious how other people think about this, and choose for their systems.

Upvotes: 0

Views: 691

Answers (1)

Salil
Salil

Reputation: 9722

You seem to be mixing multi-threading within a microservice with the microservice itself. When we say microservice, it's always about business logic. So, anything that is mere processing without business entities like consuming events from queue is a component within that microservice. If you want to scale the event consumers, you use a thread-pool.

Upvotes: 0

Related Questions