CODEWITHSUNDEEP
restmicroservices
Dorin Musteața
Dorin Musteața

Reputation: 160

Microservices "JOIN" tables within different databases and data replication

I'm trying to achieve data join between entities.

I've got 2 separated microservices which can communicate with each other using events (rabbitmq). And all the requests are currently joined within an api gateway.

Suppose my first service is UserService , and second service is ProductService.

Usually to get a list of products we do an GET request like /products , the same goes when we want to create a product , we do an POST request like /products.

The product schema looks something like this:

{
 title: 'ProductTitle`,
 description: 'ProductDescriptio',
 user: 'userId'
 ...
}

The user schema looks something like this:

{
 username: 'UserUsername`,
 email: 'UserEmail'
 ...
}

So , when creating a product or getting list of products we will not have some details about user like email, username...

What i'm trying to achieve is to get user details when creating or querying for a list of products along with user details like so:

[
 {
  title: 'ProductTitle`,
  description: 'ProductDescriptio',
  user: {
   username: 'UserUsername`,
   email: 'UserEmail'
  }
 }
]

I could make an REST GET request to UserService , to get the user details for each product.

But my concern is that if UserService goes down the product will not have user details.

What are other ways to JOIN tables ? other than making REST API calls ?

I've read about DATA REPLICATION , but here's another concern how do we keep a copy of user details in ProductService when we create a new product with and POST request ?

Usually i do not want to keep a copy of user details to ProductService if he did not created a product. I could also emit events to each other service.

Upvotes: 3

Views: 547

Answers (2)

Imran Arshad
Imran Arshad

Reputation: 4002

Approach 1- Data Replication

Data replication is not harmful as long as it makes your service independent and resilient. But too much data replication is not good either. Microservices doesn't fit well every case so we have to compromise on things as well.

Approach 2- Event sourcing and Materialized views

Generally if you have data consist of multiple services you should be considering event sourcing and Materialized views. These views are pre-complied disposable data tables that can be updated using published events from different data services . Say your "user" service publish the event , then you would update your view if another related event is published you can add/update materialized views and so on. These views can be saved in cache for fast retrieval and can be queried to get the data. This pattern adds little complexity but it's highly scale-able.

Event sourcing is basically a store to save all your events and replay the events to reach the particular state of system. Generally we create Materialized views from event store.

Say e.g. you have event store where you keep on saving all your published events. At the same time you are also updating your Materialized views. If you want to query the data then you will be getting it from your Materialized views. Since Materialized views are disposable that can always be generated from event store. Say Materialized views which was in cache got corrupted , you can completely regenerate the view from Event store by replaying the events. Say if i miss the cache hit i can still get the data from event store by replaying the events. You can find more on the following links.

Event Sourcing , Materialized view

Upvotes: 2

fose
fose

Reputation: 739

Actually we are working with data replication to make each microservice more resilient (giving them the chance to still work even if another service is down).

This can be achieved in many ways, e.g. in your case by making the ProductService listening to the events send by the UserSevice when a user is created, deleted, etc. Or the UserService could have a feed the ProductService is reading every n minutes or so marking the position last read on the feed. Etc.

There are many thing to consider when designing services and it really depends on your systems mission. E.g. you always have to evaluate the impact of coupling - if it is fine or not for a service not to be able to work when another service is down. Like, how important is a service and how is the impact on other services when this on is not able to work.

If you do not want to keep a copy of data not needed you could just read the data of the users that are related to a product. If a new product is created with a user that is not in your dataset you would then get it from the UserService. This would give you a stronger coupling then replicating everything but a weaker then replicating no data at all.

Again it really depends on what your systems is designed for and what it needs to achieve.

Upvotes: 0

Related Questions