CODEWITHSUNDEEP
microservices
amalgamat
amalgamat

Reputation: 130

How to handle common variables in microservices architecture?

Let's consider a situation, where multiple services relay on data that can change any time and should be updated in each microservice roughly at the same time - for example there is a list of supported languages or some common policies that could change one day and affect many services at once.

One solution that I could think of is to have another microservice that could hold that data and any service that needs current state can just ask for it. The drawback is that this data is not changing very frequently, asking by HTTP is not that cheap and there is a lot of traffic to this let's say global registry service. As it is not changing very often, many services could just cache the data - in order to not ask for it every time - and not be able to respond to change quick enough when the change is made to the configuration.

The other solution could be to externalize such configuration - in AWS for example there could be some configuration file on S3 that would be available for others. The drawback here is that there is no way (as far as I know) to track changes in such file and there is no way to add some logic for verification if changed value in configuration is correct (there is no typos and so on), etc.

So my question is how to handle global configuration/registry in microservice world so that there is little HTTP overhead, you can audit changes as well as introduce change at the same time in many services?

Upvotes: 5

Views: 4898

Answers (4)

mel3kings
mel3kings

Reputation: 9405

If you absolutely must have a shared config, best decoupled architecture I've encountered is as follows:

  1. You have a standalone Config Service, completely private to the outside world and can only be accessed through an internal network for your microservices
  2. ON STARTUP: Microservices do a pull request from the Config Service of what is needed per service and is stored in memory. if it is unable to pull from Config Service do not allow it to start. Have Retry Mechanism on this front.
  3. ON CHANGE of the Config Service: Publish an event to your messaging layer that will force services to update their respective configurations.

enter image description here

Caveats:

  • do not put time sensitive configurations here, since we are using asynchronous communications here (if you have time critical configs why are they shared in the first place, you might need to revisit)
  • you need to handle your own plumbing, retry mechanism, memory management etc etc.

Upvotes: 2

Mselmi Ali
Mselmi Ali

Reputation: 1257

There are couple of ways to do it, a better way especially in prod is to use external Configuration Store Pattern.

You can save the configuration in external stores like Azure Key Vault or Azure App configuration

Find more details about Azure key vault here:

Azure key vault

5-Minute quickstarts of Azure key vault integration

Upvotes: 1

ABHAY JOHRI
ABHAY JOHRI

Reputation: 2146

Use Config Server( spring cloud config server) that will maintain centralized configurations, you need to make changes to config server related to configurations, each microservices will come on startup for configurations to config server, even after start up after certain interval of time microservices can come to config server for validating any change in configurations and update accordingly.

Upvotes: 1

Anunay
Anunay

Reputation: 1893

I will prefer the option 1. Apart from the HTTP overhead, this will also lead your system in an inconsistent state. Service 1 might be working on new values but service 2 will be on old.

Since this is a distributed system that we are talking about, I am willing to take a risk with availability.

Have a configuration service that allows you to plan your config changes. Instead of saying change the value of A from x to y, you say change from x to y at time t. This t allows you to consistently propagate changes to all your system.You need to put in effort to understand what the min value of t should be for you set of services, how will you make all services acknowledge the changes and make them at the right time and how will you manage the new services that come up in between.

Another approach is use Spring Cloud Config (or something similar). It ask the service to register with the centralised config service and make refresh call to all the services to update config. Limitation being not all configs could be refreshed and if you are behind the LB you still need to handle ways to make sure all instances gets updated.

Upvotes: 3

Related Questions